>-A  2'S'c 


I  pi  I  111! 

I  III  I  II 


if  fi  i  '  p  r 
i  MB  I  ■- 


=  '  - 

in,  **  r> 

W(»y 

u 

M  m 

-  -  w 

/n) 


Adaptive  Liquid  Crystal  TV  Based  Joint  Transform  Correlator 
as  Applied  to  Real-Time  Pattern  Recognition 


Final  Beport 
By: 

7  -_  F.T.  JT.  TCU- - 

Principal  Investigator 

Department  of-  Electrical  and  Computer  Engineering' 
The  Pennsylvania  State  University 
University  Park;,  PA  16802 

Prepared  for: 

Army  Research  Office 
P.0.  Box  1221 i 

Research  Triangle  Park,  NC  27709-2211 
Attentions  Bobby  D.  Guenther 


oi 


Contract  DAAL03-87-K-0147 
ARO  Proposal  Number  25146-PH 

Period  Covered:  October  1,  1987  to  March  31,  1991 

Date:  May  23,  1991 


Is!  I!  1  !il  1 1 U 


MASTER  COPY 


KEEP  THIS  COPY  FOR  REPRODUCTION  PUr*OSES 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OM3  No  0704-0188 

Puohc  feDort'pg  O'jiden  '"is  c  fiction  cf  nfcrmat»on  s  estimated  *c  '  "our  3er  esovrse  ru.  wdmg  r-e  t*<^e  *or  '*v>e*v.nq  instructions  searching  exiting  caia  vourc« 

gathering  ana  maintaining  the  data  needed,  and  comDieting  and  reviewing  the  wCi<~.ucn  ji  ^Tcrrr.ation  >end  ents  regarding  this  cyrden  estimate  or  any  other  asoect  of  this 

cOMect-cnvf  nformation  r^ud-ng  suggestions  *or  reduc.ng  this  Durden  to  A  jsmnqtcn  «eadauarter\  jervices  directorate  for  rro"r3t  on  Ooera*  ons  and  HeocrtS  Ui5.ef*erscn 
Oavis Highway.  Suite '2Q4  Arlington.  ^  t2202^3G2  and  to  the  Office  Z*  Management  and  dudget  Paoerwcr*  ReducVcn  P-Oject  >0704-0 188)  Aasmngton  CC  23S03 

1.  AGENCY  USE  ONLY  (Leave  blank)  1  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

j  May  23,  1991  Final  Report  -  10/1/67  -  3/31/91 

4.  TITLE  AND  SUBTITLE 

Adaptive  Liquid  Crystal  TV  Based  Joint  Transform 

Correlator  as  Applied  to  Real-Iime  Pattern  Recognition 

5.  FUNDING  NUMBERS 

Df,fi 

6.  AUTHOR(S) 

Francis  T.S.  Yu 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Department  of  Electrical  and  Computer  Engineering 

The  Pennsylvania  State  University 

University  Park,  PA  16802 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

NA 

9.  SPONSORING;  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

U.  S.  Army  Research  Office 

P.  0.  Box  12211 

Research  Triangle  Park,  NC  27709-2211 

10.  SPONSORING;  MONITORING 

AGENCY  REPORT  NUMBER 

Ado 

11.  SUPPLEMENTARY  NOTES 

The  view,  opinions  and/or  findings  contained  in  this  repoi 
author (s)  and  should  not  be  construed  as  an  official  Depai 
position,  policy,  or  decision,  unless  so  designated  by  otl 

rt  are  those  of  the 
rtment  of  the  Army 
ler  documentation. 

12a.  DISTRIBUTION  /AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited. 

12b.  DISTRIBUTION  CODE 

j 

13.  ABSTRACT  (Maximum  200  words) 

The  primary  goal  of  this  research  is  to  study  a  programmable  joint-transform 
correlator  (JTC)  using  liquid  crystal  television  (LCTV)  oanels  for  adaptive  real¬ 
time  pattern  recognition  applications.  The  technique  can  improve  the  pattern 

recognition  and  identification  technology  that  is  of  interest  to  the  U.S.  Army.  The 
technique  we  studied  is  a  real-time  programmable  electro-optical  architecture.  There 
are  several  reasons  for  selecting  the  optical  technique  over  their  digital  and 
electronic  counterparts,  as  follows:  Optical  technique  is  capable  of  handling  a 
large  space-bandwidth  image;  optical  technique  is  capable  of  performing  parallel 
operations;  optics  can  perform  massive  interconnections;  optical  transformation  can 
be  operated  at  high  speed,  etc.  By  using  the  LCTV,  the  pattern  under  observation 
tan  be  correlated  with  a  large  number  of  recallable  image  memories.  In  addition, 
the  LCTV  technique  is  rather  simple  and  economical  to  operate.  The  LCTV-optical 
correlator,  in  principle,  can  be  designed  into  a  compact  portable  form  for  insitu 
application.  Brief  outlines  of  the  major  research  findings  ana  publications  are 
provided  in  this  report. 


14  subject  terms  jQ-j n£  transform  correlator,  liquid  crystal 
television  correlator,  optical  pattern  recognition 

15.  NUMBER  OF  PAGES 

1  64 

16.  PRICE  CODE 

17  SECURITY  CLASSIFICATION 

is.  security  classification 

19.  SECURITY  CLASSIFICATION 

20.  LIMITATION  OF  ABSTRACT 

OF  REPORT 

OF  THIS  PAGE 

OF  ABSTRACT 

UNCLASSIFIED 

UNCLASSIFIED 

UNCLASSIFIED 

UL 

•NSU  75-0-0'  280-5500  S'andara  -0"n  298  2  89; 


’  D**<S  D*  ~V..  »‘d 
.  *3  '  ,2 


25146-PH 


FINAL  REPORT 


1. 

2. 

o 

O  * 


4. 

D  * 
6 


ARO  PROPOSAL  NUMBER:  25146-PH 

PERIOD  COVERED  BY  REPORT:  July  1,  1988  -  March  31,  1991 
TITLE  OF  PROPOSAL:  Adaptive  Liquid  Crystal  TV  Based  Joint 

Transform  Correlator  as  Applied  to  Real-Time 
Pattern  Recognition 


CONTRACT  OR  GRANT  NUMBER:  DAAL03-87-K-0147 
NAME  OF  INSTITUTION:  The  Pennsylvania  State 
AUTHORS  OF  REPORT:  Francis  T.S.  Yu 


Univ  ersily 


CONTENTS 


Page 


7.  List  of  Publications/Manuscripts  .  1 

8.  Scientific  Personnel  . . . 5 


8.1  Scientific  Personnel  .  5 

8.2  Degrees  Awarded  . 5 


9.  Brief  Outline  of  Research  Findings 


5 


9.1 

9.2 

9.3 

9.4 

9.5 


9.6 

n  v 

o  t  • 

9.8 

9.9 

9.10 

9.11 

9.12 

9.13 

9.14 

9.15 

9.16 

9.17 

9.18 

9.19 

9.20 

9.21 

9.22 

9.23 

9.24 


Rotation  Invariant  Pattern  Recognition  .  .  . 

Effects  on  Fringe  Bir.arization . . 

Detection  Efficiencies  .  ..... 

JTC  Color  Pattern  Recognition  . 

Comparison  of  Vander  Lugt  and  Joint  Transform 

Correlators  .  . 

LCTV  C  :lor  Encoder . . 

MU  4  4-  ~  t  t 

"Illi/C  L,XgIIt  VIC  ••••»••«•**••• 

High  Efficient  JTC . . 

Autonomous  Target  Tracking  . 

Image  Deconvolution  . 

Recom. ; gurable  Interconnections . 

Binary  Phase  Correction  . 

Tmage  Classification  . 

I  PA  Neural  Network . 

LCTV  Neural  Network  . 

.N'onconventional  JTC . 

Multi-Target  Tracking  . 

Unsupervised  Learning  Neural  Net  . 

Color  Holography  Using  LiNbO,  .  , 

Compact  LCTV  Neural  Network  '  . 

Wavelength  Multiplexed  Matched  Filters  .  .  . 

Space-Time-Sharing  Neural  Network  . 

Binary  Pha-e  Only  JTC  . 

Remarks  . 


.  6 
.  7 
.  7 
.  7 

.  8 
.  8 
.  8 
.  9 
.  9 
.10 
.10 
.11 
.11 
.12 
.  12 
.13 
.13 
.14 
.  15 
.  15 
.  16 
.  17 
.  17 
.18 


References 


19 


CONTENTS  (Cont.)  P**,« 

10.  APPENDIX:  Publications . . . 22 

10.1  Rotation  Invariant  Pattern  Recognition  .  .  .22 

10.2  Effects  on  Fringe  Binarization  ...............  .26 

10.3  Detection  Efficiencies . .  .30 

10.4  JTC  Color  Pattern  Recognition  . . 39 

10.5  Comparison  Vander  Lugt  and  Joint  Transform  Correlators  ...  .44 

10.6  LCTV  Color  Encoder . .  .57 

10.7  White-Light  JTC . 63 

10.8  High-Efficient  JTC . 67 

10.9  Autonomous  Target  Tracking . 71 

10.10  Image  Deconvolution . 79 

10.11  Reconf igurable  Interconnections . 90 

10.12  Binary  Phase  Correction . . . 99 

10.13  Image  Classification . 104 

10.14  IPA  Neural  Network . .  .  .108 

10.15  LCTV  Neural  Network . 114 

10.16  Nonconventional  JTC . 118 

10.17  Multi-Target  Tracking . 122 

10.18  Unsupervised  Neural  Net . 131 

10.19  Color  Holography  Using  LiNbO, . 139 

10.20  Compact  LCTV  Neural  Network  '  .  .  .  145 

10.21  Wavelength  Multiplexed  Matched  Filters  .  .149 

10.22  Space-Time-Sharing  Neural  Network . 155 

10.23  Binary  Phase  Only  JTC . 159 


1 


LIST  OF  MANUSCRIPTS  SUBMITTED  OR  PUBLISHED  UNDER  ARO  SPONSORSHIP  DURING 
THIS  REPORTING  PERIOD,  INCLUDING  JOURNAL  REFERENCES: 

F.T.S.  Yu  and  T.  Nagata,  "Binary  Phase  only  Joint  Transform  Correlator," 
Microwave  and  Optical, Technology  Letters.  Vol.  2,  pp.  15-19,  January 
1989. 


F.  T.  S.  Yu,  S.  Jutamulia,  R.  V.  Yelaoarty  and  D.  Gregory,  "Adaptive 
Joint  Transform  Correlator  for  Real-Time  Color  Pattern  Recognition," 
Qptics..flnd  Lager  Technology.  Vol.  21,  pp.  189-192,  June  1989. 

F.T.S.  Yu,  C.  Zhang,  Y.  Jin  and  S.  Jutamulia,  "Application  of 
Holographic  Associative  Memory  to  a  Hybrid  Binary  Adder,"  3PIE 
Proceeding  on  Holographic  Optics:  Design  and^ppllcation.  Vol.  883,  pp. 
254-259,  January  1988. 

F.T.S.  Yu,  S.  Jutamulia  and  E.  Tam,  "Gray  Level  Pseudocolor  Encoding 
using  a  Liquid  Crystal  Television,"  Journal  of  Optics.  Vol.  19,  pp. 
129-123,  May-June  1989. 

F.T.S.  Yu,  F.  Cheng,  T.  Nagata,  and  Don  A.  Gregory,  "Effects  of  Fringe 
Binarization  on  Multi-Object  Joint  Transform  Correlation,"  Applied 
Optics,  Vol.  28,  pp.  2988-2990,  August  1989. 

D.  A.  Gregory,  J.  A.  Loudin  and  F.T.S.  Yu,  "Illumination  Dependence  of 
the  Joint  Transform  Correlation,"  Applied  Optics,  Vol.  28,  pp. 

3288-3290,  August  1989. 

F.T.S.  Yu,  Q.  W.  Song,  Y.  S.  Cheng  and  D.  A.  Gregory,  "Comparison  of 
Detection  Efficiencies  for  Vander  Lugt  and  Joint  Transform  Correlators," 
Applied  Optics,  Vol.  29,  pp.  225-232,  January  1990. 

F.T.S.  Yu,  X.  Li,  E.  Tam,  S.  Jutamulia  and  D.  A.  Gregory,  "Rotational 
Invariant  Pattern  Recognition  with  a  Programmable  Joint  Transform 
correlator,"  Pr.9cg.gdlnfL.of.  1988...Confgrgncg  on  Pattern  RgcortnHlon-iox 
Advance  Missile  Systems.  November  14-15,  1988. 

T.  W.  Lu,  S.  D.  Wu,  X.  Xu,  and  F.T.S.  Yu,  "A  2-D  Programmable  Optical 
Neural  Network,"  Applied  Optics,  Vol.  28,  pp.  4908-4913,  November  1989. 

F.T.S.  Yu,  X.  Y.  Li,  E.  Tam,  S.  Jutamulia  and  D.  A.  Gregory,  "Rotation- 
Invariant  Pattern  Recognition  with  a  Programmable  Joint  Transform 
Correlator,"  Applied  Optics,  Vol.  28,  pp.  4725-4727,  November  1989. 

X.  J.  Lu,  F.T.S.  Yu  and  D.  A.  Gregory,  "Analytical  Comparison  of  Vander 
Lugt  and  Joint  Transform  Optical  Correlators,"  Applied  Physics  B,  Vol. 
51,  pp.  153-164,  August  1990. 

F.T.S.  Yu,  E.  Tam,  and  D.  A.  Gregory,  "High  Efficient  Joint  Transform 
Correlator,"  Optics  Letters,  Vol.  15,  pp.  1029-1031,  September  1990. 


2 


F.i.S.  Yu,  C.  H.  Zhang,  V.  Ji:i  and  D.  A.  Gregory,  "Nonconventional  Joint 
Transform  Correlator."  Optics  Letters,  Vol.  14,  on.  922-92A,  September 
1989. 


t.i.S.  Yu,  i.  -j  in  ana  u.  Zhang ,  Symbolic  Logic  Processing  using 
Cascaded  LCTVs,"  Microwave  and  Optical  Technology  Letters,  pp.  309-313, 
September  1989. 

i.  Lu,  S.  Wu,  X.  Xu  and  F.T.S.  Yu,  'Optical  Implementation  of 
Programmable  Neural  Network,"  Proceeding  of  SPIE  on  Optical  Pattern 
Recognition.  Vol.  1053,  January  1989. 

F.i.S.  lu,  t.  Tarn,  X.  Li,  and  D.  A.  Gregory,  "Detection  of  Rotational 
and  Scale  Varying  Objects  with  a  Programmable  Joint  Transform 
Correlator,"  Proceeding  of  SPIE  on  Optical  Pattern  Recognition.  Vol. 
1053,  January  1989. 


t.  C.  Tam,  F.T.S.  Yu  and  D.  A.  Gregory,  "Target  Tracking  with  an 
Adaptive  Joint  Transform  Correlator,"  Proceedings  on  SPIE  Conference  on 
Aerospace  Pattern  Recognition.  Vol.  1098,  March  1989. 


S.  «u,  T.  W.  Lu,  X.  Xu  and  F.i.S.  Yu,  "An  Adaptive  Optical  Neural 
Network  using  a  High  Resolution  Video  Monitor,"  Microwave  and  Optical 
Technology  Letters,  Vol.  2,  pp.  252-257,  July  1989. 


T.  W.  Lu,  X.  Xu,  S.  Wu  and  F.T.S. 
Inter-Pattern  Association  ( I  PA } , " 
January  1990. 

T.  W.  Lu,  X.  X a ,  S.  Wu  and  F.T.S. 
Inter-Pattern  Association  ( I  PA ) , " 
Conference  on  Neural  Networks ,  Vo 1 


Yu,  "A  Neural  Network  Model  using 
Applied  Optics,  Vol.  29,  pp.  284-288, 


Yu i  A  Neural  Network  Model  using 
Proceed ing  oi  the  International  Joint 
.  I,  p.  596,  June  18-22,  1989. 


r.  Cheng,  X.  Xu,  S.  Wu,  F.i.S.  Yu  and  D.  A.  Gregory,  Restoration  of 
Linear-Mot ion- Blurred  Images  using  a  Joint  Transform  Processor," 
Proceeding  of  tne  SPIE  Conference  on  Optical  Information  Processing. 
Aug.  Q —  If.  1989. 


X.  i.  Li,  t.T.S.  .  and  D.  A.  Gregory,  ''Optical  Implementation  of  the 
Kittler  Transform  ir  Image  Classification,"  Proceeding  of  che  SPIE 
Conference  on  Onti  .al  Information  Processing.  Aug.  9-10,  1989. 


S.  Wjj,  Q,  Song.  A.  Mayers,  F.T.S.  Yu  and  D.  A 
Interconnections  Using  Nonlinear  Photorsfract 
Optics,  Vol.  29,  pp.  1118-1125.  March  1990. 


Gregory,  "Reconf igurable 
•‘6  Crystal",  Applied 


r.  Cheng.  X.  Xu.  S.  Wu ,  F.T.S.  Yu  and  D. 
images  Blurred  Due  to  Linear  Motion  Usin 
v icrowave  and  Optical  Technoiogv  Letters 
1990. 


A.  C-rego 
g  a  -Joint 
.  Vo  1 .  J , 


ry,  "Restoration  of 
Transform  Procession," 
pp.  21-27,  anuary 


3 


E.  Tam,  S.  Wu,  A.  Tanone,  F.T.S.  Yu  and  D.  A.  Gregory,  "Closed-Loop 
Binary  Phase  Connection  of  a  LCTV  Using  a  Point  Diffraction 
Interferometer,"  IEEE  Photonics  Technology  Letters,  Vol.  2,  pp.  143-146, 
February  1990. 

T.  Lu,  X.  Xu,  S.  Vji  and  F.T.S.  Yu,  "Hetero  Association  Neural  Network 
for  Pattern  P.ecognic  Ion;  "  Proceedings  of  1989  IEEE  International 
Conference  on  Systems.  Man,  and  Cybernetics.  November  14-17,  1989. 

E.  C.  Tam,  F.T.S.  Yu,  D.  Gregory,  ana  d.  D.  Juday,  "Autonomous  Real- 
Time  Object  Tracking  with  4d.\pti>-r  Joint  Transform  Correlator." 
Optical  Engined  lag,  Voi.  'j,  pp.  Jii- C2U ,  April  1990. 


F.T.S.  Yu,  7.  K.  Lu,  X.  Y.  Yang,  and  0.  A.  Gregory,  "Compact  Size  Neural 
Network  using  Liquid  Crystal  Televisions  "  Proceeding  of  the  SPIE 
Conference  on  Optical  Computing.  Vol.  1215,  January  1990. 


F.T.S.  Yu ,  T.  W.  Lu,  X.  Y.  Y'ang,  and  D.  A.  Gregory,  "Optical  Neural 
Network  with  Pocket-Size  Liquid-Crystal  Televisions,”  Optics  Letters. 
Vol.  15,  pp.  863-865,  August  1990. 

D.  A.  Gregory,  J.  A.  Loudin,  J.  C.  Kirsc":  ,  E.  C.  Tam,  and  F.T.S.  Yu, 
"Application  of  Hybrid  Modulating  Properties  of  the  Liquid  Crystal 
Television,"  Applied  Optics  (In  Press). 

T.  W.  Lu,  F.T.S.  Yu  and  D.  A.  Gregory,  "Self-Organizing  Neural  Network 
for  Unsupervised  Learning,"  Optical  Engineering,  Vol.  29,  pp.  1107-1113, 
September  1990. 

E.  C.  Tam,  F.T.S.  Yu,  A.  Tanone,  D.  A.  Gregory,  and  R.  Judsv,  "Data 
Association  Multiple  Target  Tracking  using  a  Phase-Mostly  LCTV,"  Optical 
Engineering.  Vol.  29,  pp.  1114-1121,  September  1990. 

F. T.S.  Yu,  E.  C.  Tam,  D.  A.  Gregory,  ar.d  R.  Juday,  "Multi -target 
Tracking  with  a  Hybrid  Joint  Transform  Correlator, ”  Proceeding  of  the 
SPIE  Conference  on  Real-Tise  Image  Processing  II.  Vol.  1295.  pp.  1 28— 
137,  April  1990. 

T.  ui,  F.T.S.  Yu,  and  D.  A.  Gregory.  "Optical  Neural  Network  for 
Unsupervised  Learning,"  proceeding  of  the  SPIE  Conference  cn  Advances  i r 
Optical  information  Processing  IV,  Vol.  1296,  April  1990. 


F.T.F.  Yu.  ~.  Lu  an.i  X.  Yang,  "Optical  Implementation  of  Hetero- 
Association  Neural  Network  with  ; nter-Fattern  Association  Model." 
International  Journal  of  Optical  Computing  {In  Press). 


F.T.S.  Yu,  X.  Yang  and  fu,  "Space-Time  Snaring  Optical  Neural 
Network."  Ontics  Letters,  voi.  16.  op.  247-249,  February  (1991). 


A 

T 


J.  C.  Kirsch,  D.  A.  Gregory,  J.  A.  Loudin,  £.  C,  Taa  and  F.T.S.  Yu, 
''Light  Efficient  Joint  Transform  Optical  Correlator,"  Proceedings  of  the 
SPiE  Conference  on  Optical  Information  Processing  Systems  and 
Architectures  II,  San  Diego,  CA,  July  8-13,  1990. 

F.T.S.  Y'u,  T.  Lu  and  D.  A.  Gregory,  "Compact  Optical  Neural  Network 
using  Liquid  Crystal  Television,"  Proceeding  of  the  IC0-i5th  Conference 
on  Optics  in  Complex  Systems.  Garmisch,  Germany,  SPIE  Voi.  1319,  pp. 
200-201,  August  1990. 

F.T.S.  Y'u  and  F.  c.  Tnm(  High  Efficiency  JoAuL  Transform  correlator 
for  Symbolic  Substitution,"  Proceeding  of  the  I  CO-15  Conference  on 
Optics  in  Complex  Systems.  Garmisch,  Germany,  August  5-10,  1990 

F.T.S.  Y’u.  "Optical  Neural  Network:  Architecture,  Design  and  Models," 
Invited  Proceeding  of  IEEE  TENCON  *90  Conference  on  Computer  and 
Communication  Systems,  Hong  Kong,  pp.  12-16,  September  24-27,  1990. 

F.T.S.  Y'u  and  T.  Lu,  "Adaptive  Optical  System  for  Neural  Computing," 
Proceeding  of  IEEE  TENCON  ’90  Conference  on  Computer  and  Communication 
Systems,  Hong  Kong,  September  24-27,  1990. 

F.T.S.  Y'u,  S.  Ku,  A.  Mayers,  S.  Rajan  and  D.  A.  Gregory.  "Color 
Holographic  Storage  in  LiNbO,,"  Optics  Communications,  Vol.  81,  pp.  343- 
352,  March  1991. 

F.T.S.  Y’u,  S.  Ku,  A.  Mayers,  S.  Rajan,  "Wavelength  Multiplexed 
Reflection  Matched  Spatial  Filters  using  LiNbO. to  Cpti.cs 
Communications,  Vol.  81,  pp.  343-347,  March  19M. 

F.T.S.  Y'u.  T.  h.  Lu  and  D.  A.  Gregory,  "Self-Learning  Optical  Neural 
Network , "  Proceedings  of  Spatial  Light  Modulators  snd  Applications , 
Technical  digest,  Vo5.  14,  pp.  24-27,  September  1990- 

F.T.S.  Y'u,  Ku,  A.  Mayers,  S.  Rajan  and  D.  A.  Gregory,  "s"oIor 
Holographic  Storage  in  LiNLO. , "  Proceedings  of  Spatial  Llghu  Modulators 
and  Applications.  Technical  bigest,  Vol.  14,  pp.  153-1.56.  September 
1990. 

X.  Y'ang,  T.  Lu,  i-  T.3.  Yu  r.nd  D.  A.  Gregory,  "Redundant  Interconnection 
Interpattern  Association  Neural  Network,"  Applied  Optics  (In  Press.'. 

F.T.S.  Y’u,  Y.  Li,  X.  Y’ang.  T.  Lu  and  D.  A.  Gregory,  "Application  of 
Moment  Invariant  Pattern  Recognition  to  Optical  Neural  Net,"  submitted 
to  Optik, 


1 


SCIENTIFIC  PERSONNEL  SUPPORTED  BY  THIS  PROJECT  AND  DEGREES  AWARDED 
DURING  THIS  REPORTING  PERIOD: 


8.1  Scientific  Personnel 

F.T.S.  Yu  -  Principal  Investigator 
A.  Mayers  -  Research  Assistant 

E.  Taa  -  Research  Assistant 

X.  Li  -  Research  Assistant 

F.  Chens  -  Research  Resistant 

A.  Tanone  -  Research  Assistant 


S.  Raj an  -  Research  Assistant 

T.  W.  Lu  -  Research  Assistant 

8.2  Degrees  Awarded 
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"Applications  of  Moment  Invariants  to  Neuroconputing  for  Pattern 
Recognition,"  Yajun  Li,  Ph.D.,  coapleted  June  1990. 

"Modulation  properties  of  a  twisted  nematic  liquid  crystal  spatial 
light  modulator  and  its  applications  in  a  joint  transfora 
correlator,"  E.  C.  Tao,  Ph.D.,  coapleted  August  1990. 

"Study  of  an  optical  neural  network. .. architecture,  design  and 
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"Optical  image  classification  using  opticai/digitai  hybrid  image 
processing  system,"  J.  X.  Li,  completed  October  1990. 


9.  BRIEF  OUTLINE  OF  RESEARCH  FINDINGS 


In  a  conventional  coherent  optical  signal  processor  [1],  the  processing 
operation  is  usually  carried  out  at  the  spatial  frequency  or  Fourier  plane 
with  a  complex  spatial  filter  [2].  This  type  of  coherent  optical  processor 
offers  a  myriad  of  complicated  processing  operations  [3].  Its  success  is 
primarily  due  to  the  profound  diffraction  phenomena.  However,  complex  signal 
processing  can  also  be  achieved  by  the  spatial  impulse  response  using  a  joint 
transform  processor.  There  are  several  inherent  advantages  of  using  the  joint 
Fourier  transform  processor  as  compared  with  the  conventional  coherent 
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processor:  (1)  spatial  filter  adjustment  is  not  imposed;  (2)  a  higher  input 
space-bandwidth  product;  (3)  generally,  a  higher  modulation  index  of  the  joint 
transform  hologram;  (4)  lower  spatial  carrier  frequency,  etc.  In  view  of 
these  advantages,  a  joint  transform  processor,  in  principle,  is  capable  of 
performing  optical  signal  processing  more  efficiently,  particularly  in  the 
application  of  real-time  pattern  recognition.  The  purpose  of  this  research 
program  is  to  investigate  an  adaptive  liquid  crystal  TV  based  correlator  as 
applied  to  real-time  pattern  recognition  and  tracking. 

In  this  period,  from  July  1,  1988  to  March  31,  1991,  we  have 
accomplished  several  major  tasks  on  the  research  of  real-time  pattern 
recognition  with  an  LCTV  based  correlator  for  which  various  results  have  been 
reported  in  the  refereed  journals  and  conference  proceedings;  with  the 
approval  of  the  Department  of  the  U.S.  Army.  Sample  copies  of  these 
publications  are  included  in  this  final  technique  report  in  the  subsequent 
sections,  to  provide  a  concise  documentation  of  our  findings.  In  the 
following  sections,  we  shall  give  an  overview  of  our  research  work  done  during 
this  program.  We  will  highlight  some  of  the  accomplished  works.  A  list  of 
publications  resulting  from  this  support  has  been  cited  in  the  preceding 
section. 

9.1  Rotation  Invariant  Pattern  Recognition  (APPENDIX  10.1) 

During  this  research  program,  we  have  successfully  investigated 
rotational  invariant  pattern  recognition  using  a  programmable  liquid  crystal 
television  (LCTV)  based  joint  transform  correlator  (JTC).  We  have  introduced 
two  methods  to  recognize  targets  with  different  in-plane  rotational 
orientations.  One  method  is  using  real  value  implementation  of  circular 
harmonic  expansion  techniques  for  joint  transform  target  detection,  the  other 
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experimental  demonstrations  for  these  two  methods  have  ten  Der formed.  A 
paper  of  this  nature  is  published  in  Applied  Option  {4]. 

5.2  Effects  on  Fringe  Binarizatic-n  (APPENDIX  10.2) 

we  have  also  studied  tne  effects  of  fringe  binarization  on  au 1 1 1 -oh  j ect 
joint  transfers  correlation,  we  have  sho-n  that  binarization  joint  transform 
interference  fringes  for  multiple  targets  would  produce  false  alarms  and 
misses.  «e  nave  concluded  that  fringe  binarization  aay  net  be  usefully 
applied  for  aulti-carget  recognition,  although  it  can  be  applied  successfully 
for  noisefree  siRger  target  correlation.  A  letter  to  the  editor  of  these 
findings  was  published  in.  Applied  Optics  (5j. 

9.3  Detection  Efficiencies  (APPEHDIX  10.3} 

In  this  period  of  research,  we  investigated  the  detection  efficiencies 
for  Vander  Lugt  and  joint  transform  correlators.  We  have  found  that  the 
relative  performance  of  a  Vander  Lugt  correlator  would  generally  perform 
better  in  tne  multi-object  and  under  noisy  environment.  These  effects  are 
caused  by  the  signal  dependent  nature  of  the  joint  transform 
the  Vander  Lugt  filter,  it  is  signal  indeps 
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spectral  content  of  the  target.  A  paper  of  this  nature  is  published  in  the 
Optics  and  Laser  Technology  [7]. 

9.5  Comparison  of  Vander  Lugt  and  Joint  Transform  Correlator  (APPENDIX  10.5) 
We  completed  a  theoretical  study  of  comparing  the  Vander  Lugt  and  joint 

transform  correlators.  Multiplicative  noise  and  differences  between  input  and 
reference  functions  are  considered  so  that  their  effects  on  optical 
correlation  can  be  estimated.  We  have  found,  in  general,  the  JTCs  are  easier 
to  construct,  simpler  to  handle  and  control  and  more  practical  for  application 
than  Vander  Lugt  correlators.  However,  the  basic  distinction  between  these 
two  correlators  are:  the  JTC  filter  is  signal  dependent,  whereas  the  Vander 
Lugt  filter  is  signal  independent.  A  paper  of  this  study  is  published  in 
Applied  Physics  [8]. 

9.6  LCTV  Color  Encoder  (APPENDIX  10.6) 

While  studying  the  polarization  properties  of  the  liquid  crystal 
materials,  we  have  developed  a  technique  to  perform  real-time  gray  level 
pseudocolor  encoding  using  a  LCTV.  The  LCTV  is  used  under  two  different 
polarizer/analyzer  combination  to  generate  a  positive  and  a  negative  image. 
These  images  are  then  encoded  with  two  primary  colors  by  positioning  the  color 
filter  respectively.  The  superposition  of  these  images  would  produce  a  gray 
level  pseudocolor  encoded  image.  The  result  of  this  study  has  been  published 
in  Journal  of  Optics  [9]. 

9.7  White-Light  JTC  (APPENDIX  10.7) 

In  this  period  of  research,  we  have  also  developed  a  technique  of 
performing  joint  transform  correlation  using  a  white-light  source.  The 
temporal  and  spatial  coherence  requirement  of  a  joint  transform  correlator 
(JTC)  have  been  studied.  Excellent  experimental  results  have  been  obtained. 


9 


Since  the  white-light  source  emanates  ail  visible  spectra,  the  white-light  JTC 
is  capable  of  exploiting  the  spectral  content  of  the  target.  A  paper  of  this 
nature  has  been  published  in  Optics  Letters  [10]. 

5.8  High-Efficient  JTC  (APPENDIX  10.8) 

We  have  developed  a  high  efficient  JTC.  By  modifying  the  joint 
transform  power  spectrum  of  the  input  objects,  using  a  spectral  sampling 
method,  the  read  out  light,  as  well  as  the  physical  area,  of  the  square  low 
detector  can  be  fully  utilized.  As  a  result,  the  output  correlation  intensity 
can  be  substantially  increased.  This  technique  would  benefit  the  real-time 
optical  pattern  recognition  performance.  To  improve  the  signal-to-noise,  the 
read-out  can  also  be  done  by  using  partially  coherent  light  A  paper  of  this 
report  is  published  in  Optics  Letters  [11]. 

9.9  Autonomous  Target  Tracking  (APPENDIX  10. S) 

One  of  (.he  most  interesting  research  tasks  we  have  accomplished  in  this 
period  is  the  application  of  liquid  crystal  TV  band  JTC  to  autonomous  target 
tracking.  The  fundamental  element  of  the  system  is  a  real-time  optical  joint 
transform  correlator  using  a  liquid-crystal  television.  The  parallelism,  high 
processing  speed,  and  adaptive  properties  of  this  optical  system  assure  high 
correlation  between  objects  in  two  sequential  frames.  The  relative  position 
of  the  object  can  then  be  determined  based  on  the  location  of  the  correlation 
peak.  System  performance  is  elevated  and  experimental  demonstrations  have 
been  conducted.  we  stress  that  this  technique  has  important  applications  to 
real-time  missile,  aircraft  and  vehicle  tracking,  which  are  very  significant 
for  the  L'.S.  Army  strategic  defense  needs  in  guided  weapons  and  high  speed 
target  tracking  systems.  We  have  reported  a  paper  of  this  research  in  Optical 
Engineering  f  12] . 
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9.10.  Image  Deconvolution  (APPENDIX  10.10) 

In  this  phase  of  research,  we  have  also  performed  image  deconvolution 
due  to  linear  motion  using  a  1-D  joint  transform  correlator.  The  technique  is 
basically  using  a  blur  function,  as  the  reference  object,  in  a  joint  transform 
correlator  to  reverse  the  negative  phase  spectrum  of  the  blurred  image.  In 
other  words,  the  method  is  equivalent  to  using  a  deblurring  phase  filter  in  a 
conventional  coherent  processor.  The  advantages  of  this  technique  are 
simplicity  and  flexibility,  for  which  blurred  images  can  be  corrected  by  using 
real-time  devices.  A  paper  reports  the  image  deblurring  using  a  1-D  JTC 
architecture  was  published  in  Optics  Communications  [13],  and  a  paper  deals 
with  the  restoration  of  linear  smeared  image  is  published  in  Microwave  and 
Optical  Technology  Letters  [14], 

9.11  Reconf igurable  Interconnections  (APPENDIX  10.11) 

High  speed  computing  has  aroused  great  interest  xn  optical 
interconnections.  We  have,  in  this  period,  investigated  a  reconf igurable 
optical  interconnection  using  a  nonlinear  photoref ractive  crystal.  We  have 
used  the  coupled  wave  theory  and  law  of  refraction,  to  analyze  the  volume  of 
holograms.  Reconf igurable  interconnections  are  discussed  employing  either 
wavelength  tuning  or  spatiai  division  technique.  Reflection  type  volume 
holograms  can  be  used  for  a  large  number  of  reconf igurable  interconnections  in 
terms  of  finite  wavelength  tunability.  Transmission  volume  holograms  encoded 
in  pinhole  holograms  can  be  easily  reconfigurated  by  SLM  for  optical 
interconnections.  Experimental  demonstrations  ha\ e  also  been  conducted  in 
this  phase  of  research.  We  stress  that,  the  proposed  reconf igurable 
interconnection  technique,  would  have  significant  impact  on  the  future 
r-'cearch  of  high  speed  optical  computing.  We  have  reported  a  paper  to  the 
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Applied  Optics  of  this  study  [15]. 

9.12  Binary  Phase  Correction  (APPENDIX  10.12) 

In  this  research  program,  binary  phase  correlation  of  a  liquid  crystal 
television  (LCTV)  using  a  point  diffraction  interferometer  has  been 
investigated.  The  use  of  a  point  diffraction  interferometer  offers  the 
simplicity  and  advantages  of  a  common-path  interferometer  for  examining  the 
phase  non-uniformity  of  the  LCTV,  while  the  on-line  and  closed-loop 
architecture  is  capable  of  generating  the  optimum  phase  correction  function. 
Both  computer  simulations  and  experimental  results  demonstrate  the 
effectiveness  of  the  binary  phase  correction  scheme.  The  results  obtained 
from  the  bipolar  phase  only  joint  transform  correlation  reveals  the 
feasibility  of  this  phase  correction  technique  in  actual  applications. 

Mention  must  be  made  that  this  binary  phase  correction  scheme  can  be  used  to 
compensate  for  the  phase  distortion  of  a  general  optical  signal  processing 
system.  Other  phase  modulating  real-time  addressable  SLMs  which  have  a  higher 
contrast  ratio  and  a  larger  space-bandwidth  product,  such  as  the  magnetooptic 
SLMS  or  liquid  crystal  light  valves  may  be  employed  to  produce  even  better 
results.  A  paper  reporting  these  findings  is  published  in  the  IEEE  Photonics 
Technology  Letters  [16]. 

9.13  Image  Classification  (APPENDIX  10.13) 

We  have  used  a  joint  transform  correlator  for  the  study  of  image 
classification  by  the  Kittler-Young  transform.  The  Kittler-Young  transform, 
is  a  nonparametric  method  for  feature  extraction,  which  can  be  effectively 
applied  to  image  classification.  The  advantage  of  using  the  K-Y  transform  is 
that  it  goes  one  step  further  in  utilizing  the  eigenvalues.  The  eigenvalues 
are  separated  into  two  parts,  which  are  proportional  to  the  class  means  and 
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variances  respectively.  The  effects  on  the  feature’s  discriminative  power  are 
in  opposing  ways.  To  overcome  the  difficulty  caused  by  this  contradiction,  a 
feature  selection  criterion  based  solely  on  the  class  means  after  normalizing 
class  variances  is  used  with  the  K-Y  transform.  Some  of  the  limitations  of 
using  the  first  three  algorithms  can  be  alleviated  using  the  K-Y  transform, 
resulting  in  a  more  discriminative  feature  space.  Experiments  to  classify 
birds  and  fish,  using  the  K-Y  transform  are  demonstrated.  The  results  show 
that  the  K-Y  transform  offers  a  high  feature  selection  than  using  the 
Fukunaga-Koontz  transform.  We  have  published  a  paper  of  these  findings  in  the 
Micro.  Opt.  Tech.  Lett.  [17]. 

9.14  I  PA  Neural  Network  (APPENDIX  10.14) 

During  this  research  program,  we  investigated  a  neural  network  model 
using  Inter-Pattern  Association  (IPA).  The  IPA  model  uses  basic  logical 
operations  to  determine  the  inter-pattern  association  (i.e.,  association 
between  the  reference  patterns),  and  simple  logical  rules  are  applied  to 
construct  tri-state  interconnections  in  the  network.  Computer  simulations  for 
the  reconstruction  of  similar  English  letters  embedded  in  the  random  noise  by 
IPA  model  have  shown  improved  performance  in  comparison  with  the  Hopfield 
model.  A  2-D  hybrid  optical  neural  network  is  used  to  demonstrate  the 
usefulness  of  the  IPA  model.  Since  there  are  only  three  gray  levels  used  in 
the  Interconnection  Weight  Matrix  (IWM)  for  IPA  model,  the  dynamic  range 
imposed  on  a  Spatial  Light  Modulator  (SLM)  is  rather  relaxed,  and  the 
interconnections  are  much  simpler  than  the  Hopfield  model.  A  paper  of  this 
study  is  published  in  Appl.  Optics.  [18]. 

9.15  LCTV  Neural  Network  (APPENDIX  10.15) 

We  have  developed  an  8x8  neuron  optical  neural  network  using  pocket  size 
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liquid  crystal  televisions  (LCTVs).  The  size  of  the  optical  neural  network  is 
about  75x15x25  cm”.  The  operation  speed  of  this  LCTV  neural  network,  with  8x8 

t 

neurons  is  about  1.2x10”  interconnection  operations/sec.  However,  for  a  16x20 
neural  work,  the  operation  speed  can  be  as  high  as  3xl0”!  inter. oper. /sec. 
Experimental  results  from  this  LCTV  neural  network  are  demonstrated.  A  paper 
of  this  work  is  published  in  Optics  Letters  [19]. 

9.16  Nonconvent ional  JTC  (APPENDIX  10.16) 

In  this  research  program,  we  have  also  investigated  a  nonconvent ional 
joint  transform  correlator  (NJTC).  The  advantages  of  the  NJTC  are  the 
efficient  use  of  the  light  source,  the  use  of  smaller  transform  lenses,  higher 
correlation  peaks,  and  a  higher  carrier  fringe  frequency.  Since  NJTC  can,  in 
principle,  process  all  the  information  that  a  conventional  optical  processor 
can  offer,  we  believe  that  the  NJTC  would  have  all  the  processing  capabilities 
of  a  coherent  processor.  A  paper  of  this  study  is  published  in  Optics  Letters 
[20]. 

9.17  Multi-Target  Tracking  (Appendix  10.17) 

One  of  the  major  advantages  of  optical  signal  processing  is  the 
parallelism  in  handling  high  space-bandwidth  signals.  However,  the  price  paid 
is  the  complexity  and  stringent  alignment  of  an  optical  system.  On  the  other 
hand,  its  digital  counterpart  offers  flexibility  and  programmability  while 
sacrificing  full  parallelism,  vre  have  investigated  a  technique  of  usi-g  data 
association  target  tracking  in  a  motion  sequence  via  an  adaptive  join,, 
transform  correlator.  The  massive  data  in  the  field  of  view  can  be  reduced  to 
a  iew  correlation  peaks.  The  average  velocity  of  a  target  during  the  tracking 
cycle  is  then  determined  from  the  location  of  the  correlation  peak.  We  have 
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used  a  data  association  algorithm  for  the  analysis  of  these  correlation 
signals,  for  which  multiple  targets  can  be  tracked.  Simplicity  in  optical 
architecture,  relaxation  of  alignment  requirements,  and  the  adaptive  property 
are  the  major  advantages  of  a  hybrid  JTC  in  this  mode  of  operation.  The  phase 
modulation  property  of  a  LCTV  has  been  investigated  and  was  employed  in  this 
experiment  to  improve  the  detection  efficiency  of  a  JTC.  A  motion  sequence  of 
three  targets  was  used  as  the  input  scenes  to  demonstrate  the  applicability  of 
this  technique  and  satisfactory  results  were  obtained.  A  paper  reporting  the 
performance  of  this  technique  is  published  in  Optical  Engineering  [21]. 

9.18  Unsupervised  Learning  Neural  Net  (APPENDIX  10.18) 

One  of  the  features  in  neural  computing  must  be  the  adaptability  to 
changeable  environment  and  to  recognize  unknown  objects.  We  have,  in  this 
period  of  research,  implemented  Kohonon’s  self-organizing  feature  map  for 
unsupervised  learning  in  an  optical  neural  network.  A  compact  optical  neural 
network  of  64  neurons  using  liquid  crystal  televisions  is  used  for  this  study. 
To  test  the  performances  of  the  self-organizing  neural  network,  experimental 
demonstrations  with  computer  simulations  are  conducted.  Effects  due  to 
unsupervised  learning  parameters  have  been  analyzed.  It  is  shown  that  the 
optical  neural  network  is  capable  of  performing  both  unsupervised  learning  and 
pattern  recognition  operations  simultaneously,  by  setting  two  matching  scores 
in  the  learning  algorithm.  We  have  also  implemented  the  fast  and  slower 
learning  models  in  the  neural  networks.  We  have  found  that  by  using  a  slower 
learning  rate  the  construction  of  the  memory  matrix  becomes  topologically  mure 
organized.  Moreover,  by  introducing  the  forbidden  regions  in  the  memory 
space,  it  would  enable  the  neural  network  to  learn  new  patterns  without 
erasing  the  old  ones.  A  paper  describing  these  findings  is  Dublished  in 
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Optical  Engineering  [22]. 

9.19  Color  Holography  Using  LiNbO,  (APPENDIX  10.19) 

Two  of  the  most  widely  used  white-light  holograms  must  be  the  reflection 
hologram  of  Denisyuk  and  the  rainbow  hologram  of  Benton.  In  reflection 
hologram,  a  thickness  emulsion  of  about  20pm  would  have  a  wavelength 
selectivity  about  Ai/.  =  1/40,  which  is  high  enough  to  produce  color  hologram 
images  without  significant  color  blur.  However,  the  physical  requirements  for 
constructing  a  reflection  hologram  is  rather  stringent,  which  prevents  its 
wide  spread  use  of  applications.  On  the  other  hand,  construction  of  a  rainbow 
hologram  requires  a  narrow  slit,  for  which  the  parallax  information  of  the 
hologram  image  would  be  partly  lost.  In  this  research  program,  we  have  also 
demonstrated  that  color  holograms  can  be  constructed  in  a  photorefractive 
crystal  using  a  thick  LiNbO.  crystal  with  a  "white-light”  laser.  Since 
photorefractive  crystal  is  much  thicker  than  conventional  photographic 
emulsion,  it  provides  a  higher  wavelength  selectivity  such  that  the  color  blur 
can  be  minimized.  Furthermore,  the  construction  of  photorefractive  holograms 
is  in  real-time  mode  and  the  shrinkage  of  the  emulsion  can  be  prevented.  As 
in  contrast  with  the  photographic  film,  multiplexing  c jlor  holograms  in  a 
photorefractive  crystal  is  feasible  for  which  we  have  experimentally 
demonstrated.  We  have  shown  ';hat  to  minimize  the  cr’or  crosstalk,  a  high 
wavelength  selectivity  { i - e . ,  thick  crystal)  photorefrac;  media  than 
normally  required  should  be  used.  A  paper  reporting  tbi  color  holographic 
imaging  using  a  white-light  laser  is  published  in  Optics  Communications  [23]. 

9.20  Compact  LCTV  Neural  Network  (APPENDIX  10.20) 

In  this  research,  we  have  also  developed  a  compact  optical  neural 
network  using  two  tightly  cascaded  liquid  crystal  televisions  (LCTVs).  This 
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new  optical  architecture  offers  the  compactness  in  size,  easy  alignment, 
higher  light  efficiency,  better  image  quality,  and  low  cost.  The 
implementation  of  the  auto-associative  and  hetero-associative  memories  to  this 
new  architecture  are  also  tested.  The  most  import  •  o?ect  of  this  new 

architecture  i:j  that  a  large  numerical  aperture  ;  '  w'  1  •  s  used  in  the 

previous  optic'.l  neural  network  can  be  avoided.  '!».*  a  more  compact  size 

system  can  be  milt.  Moreover,  the  lightly  cascade  -  of  the  system  also 

relax  the  fine  alignment  problem  posed  by  the  opt*  .  system.  This  proposed 
architecture  wi ch  experimental  demonstration  is  published  in  Applied  Optics 
[24]. 

9.21  Wavelength  Multiplexed  Matched  Filters  (APPENDIX  10.21) 

W>  have  studied  the  wavelength  multiplexing  capability  using  a  thick 
LiNbO,  for  reflection  matched  spatial  filter  synthesis.  In  principle,  over 
1000  spatial  filters-  can  be  multiplexed  in  a  4mm  thick  photorefractive 
crystal.  The  wavelength  multiplexing  technique  in  a  crystal  filter  has  the 
advantage  that  neither  the  object  nor  the  reference  beam  needs  to  be  changed 
in  the  writing  or  reading  processes,  even  though  the  two  processes  can  be  used 
in  conjunction  to  further  increase  the  multiplexing  capabilities.  Although 
this  technique  has  been  used  in  holographic  film  which  matched  filters  were 
multiplexed,  the  thin  film  emulsion  leads  to  low  wave-length  selectivity,  and 
the  filters  are  difficult  to  produce.  By  referring  the  coupled-mode  analysis, 
we  have  analyzed  the  wavelength  multiplexing  capability  of  reflection  type 
matched  spatial  filters.  By  integrating  over  the  thickness  of  the  crystal,  we 
have  show,;  that  thick  crystal  filters  exhibit  high  wavelength  selectivity  as 
well  as  retaining  a  good  degree  of  the  shift  invariance.  Thus,  by  recording 
single  wavelength  filters,  and  reading-out  the  filters  with  a  multi-wavelength 
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source,  parallel  pattern  recognition  can  be  performed.  A  paper  on  this  study 
is  published  in  Optics  Communications  [25]. 

9.22  Space-Time-Sharing  Neural  Network  (APPENDIX  10.22) 

To  provide  the  neural  network  with  learning  ability,  spatial  light 
modulators  (SLMs),  such  as  liquid  crystal  television  (LCTV),  have  been  used  as 
the  programmable  devices.  In  a  fully  interconnect  neural  network,  every 
neuron  at  the  input  plane  is  connected  to  all  the  neurons  at  the  output  plane. 
For  example,  1000  neurons  would  require  a  million  interconnections.  Thus,  a 
very  high  resolution  SIM  is  required  for  the  massive  interconnection. 

However,  the  resolution  of  the  currently  available  LCTVs  is  rather  limited, 
which  poses  an  obstacle  for  large  scale  operation  we  have,  in  this  phase  of 
research,  used  a  space-time  sharing  technique  to  alleviate  this  constraint. 
Since  the  LCTV  neural  networ*.  is  essentially  a  white-light  processing  system, 
by  partitioning  the  interconnection  weight  matrix  into  an  array  of 
submarines,  s.  large  sr ace-bandwidth  pa  ,er  (SBP)  can  be  processed  with  a 
smaller  neural  networx.  We  have  shown  that  to  achieve  a  large  SBP  of  the 
system,  additional  expenditure  of  processing  time  is  needed.  The  amount  of 
processing  time  increases  as  the  square  function  of  the  SBP  of  the  input 
pattern.  We  have  also  experimentally  demonstrated  that  a  12x12  and  24x24 
element  pattern  can  be  indeed  on  a  6x6  LCTV  neuron  network.  A  paper  of  this 
study  is  published  in  Optics  Letters  [26]. 

9.23  Binary  Phase  Only  JTC  (APPENDIX  10.23) 

We  have  also  in  the  research  investigate  a  joint  transform  hybrid 
optical  correlator  as  applied  to  binary  phase  only  filtering.  The 
interference  tringes  of  the  joint  Fourier  transformation,  due  to  the  reference 
and  the  input  targets,  are  captured  by  a  CCD  camera  feeding  into  a 
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microcomputer  for  binarization.  A  binarized  phase  only  filter,  in  principle, 
can  be  written  onto  a  spatial  light  modulator  (SLM).  By  simple  coherent 
illumination  of  SLM,  correlation  operation  of  the  reference  and  the  input 
targets  can  be  obtained  computer  simulated  results  for  target  embedded  in  an 
additive  noise  is  provided.  We  have  shown  that  high  correlation  peaks  can  be 
readily  achieved  under  noisy  environment.  Since  JTC  filter  is  signal 
dependent,  for  multi-target  recognition,  it  would  produce  false  alarms  and 
misses.  A  paper  of  this  nature  is  published  in  Micro.  Opt.  Tech.  Lett.  [27]. 
9.24  Remarks 

The  ability  to  process  a  large  quantity  of  information  at  a  high  speed 
makes  the  optical  correlator  an  attractive  candidate  for  applications  to 
machine  vision,  target  tracking  and  detection,  etc.  Although  conceptually 
simple,  the  Vander  Lugt  correlator,  which  employs  a  holographic  spatial 
filtering  technique,  has  inherent  filter  synthesis  and  alignment  problems, 
that  prevent  its  wide  spread  practical  application.  On  the  other  hand,  the 
Joint  Transform  Correlator  (JTC)  is  a  simple  and  practical  processor  that 
overcomes  these  two  major  disadvantages.  In  this  report,  we  have  performed 
the  major  tasks  proposed  on  the  story  of  the  LCTV  joint  transform  correlator 
and  other  LCTV  based  system,  as  applied  for  pattern  recognition.  We  are 
confident  that  our  goal  of  realizing  a  practical  adaptive  LCTV  based  JTC  for 
real-time  recognition  and  identification  would  happen  in  the  near  future. 
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A  circular  harmonic  expansion  filter  is  real  value  imple¬ 
mented  in  a  joint  transform  correlator  architecture  to  per¬ 
form  rotation  invariant  pattern  recognition. 


One  of  the  most  common  problems  in  pattern  recognition 
is  to  identify  objects  of  different  sizes,  angular  orientations 
and  perspectives.  Utilizing  the  architecture  of  a  joint  trans¬ 
form  correlator  ‘JTC’1-3  as  shown  in  Fig.  1,  we  propose  to  uce 
the  real  value  implementation  of  circular  harmonic  functions 
as  references  in  a  JTC  to  perform  rotation  invariant  pattern 
recognition. 

If  a  2-D  function  f(r,9)  is  continuous  and  integrable  over 
the  region  (0,2?r),  it  can  be  expanded  into  a  Fourier  series  as 
+  «• 

/(r,0)=  7  FJr)  exp(im9),  (1) 

mm  -«* 

where 

Fm(r)  =  [  f(r,0)  exp (~im9)dd.  (2) 

in  j0 

Fm(r,6)  -  Fm(r)  exp(imff)  is  called  the  mth-order  circular 
harmonic. 

Previous  research4-7  has  shown  that  by  using  a  single  har¬ 
monic  Fk(r,a )  as  a  complex  reference  function,  the  intensity 
of  the  correlation  center  is  independent  of  the  rotation  angle 
a.  Although  f(r,6)  is  a  complex  function  in  general,  it  is 
usually  represented  by  a  real  quantity  in  practice.  Thus  Eq. 
(1)  can  be  written  as 

+  » 

f(r,0)  =  F„(r)  +  2  V  Re|Fm(r)  exp(tm0)] 

l»l*l 


=  Fn(r)  +  2  V  |F„,(r)!  cos(mtf  +  4>m),  (3) 


where  Fm(r)  =  |Fm(r)|exp(i$m).  In  this  real  value  implemen¬ 
tation  technique,  we  define  the  following  two  reference  func¬ 
tions: 

R*,(r,0)  =  2|F*(r)|  cos(k$  +  <pk),  (4) 

R *2(r,0)  =  2iF*(r)l  sin (k$  +  <J>h).  (5) 

These  real  value  functions  are  basically  the  real  and  imagi¬ 
nary  parts  of  the  circular  harmonic  F*(r,6).  Using  Rm  as  the 
reference  image  function  and  f(r,6  +  a)  as  the  input  target 
function,  the  center  value  of  the  cross  correlation  becomes 

Cm  »  fr  f’  f{rfi  +  c\Rk]rdrde.  (6) 

Jo  Jo 

By  substituting  Eqs.  (4)  and  (5)  into  Eq.  (6),  we  have 


Cm(«) 


mF0(r)  +  2  f  iFJrJl 

m-1 

X  cos(m(0  +  a)  +  4>m)j 


X  |2lF*U  )l  cos(kff  +  4>k)lrdrdS.  (7) 

Since  all  the  terms  in  Eq.  (7)  vanish  except  the  m  =  k  term, 
the  above  equation  can  be  reduced  to 

Cm(<*)  =  4 it  cos(fea)  [  rlF*(r)pdr 
Jo 


-  2Ct(0)  cos(ka). 


(8) 


Fig.  1.  Real-time  programmable  joint  transform  correlator:  L 1 
and  L 3  are  collimating  lenses,  L2  and  lA  arc  Fourier  transform 
lenses,  ana  HS  is  a  beam  splitter. 
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Fig.  2.  Input  image  used  in  the  correlation  experiment.  The  real 
and  imaginary  circular  harmonic  reference  images  are  displayed  at 
the  upper  part,  and  the  input  target  is  at  the  lower  part. 


\  ' 


Fig.  3.  Experimental  result  obtained  from  the  JTC. 


The  corresponding  intensity  is 

lk]{a)  =  4Cji(0)  cos2(fea),  (9) 

which  varies  as  a  cosine  square  with  respect  to  the  target 
orientations.  Likewise,  if  R*2(r\0)  is  used  as  the  reference 
image  function,  the  intensity  of  the  correlation  center  is 

/*,(«)  =  4Cjf(0)  sitrl&nr).  (10) 

Thus,  by  adding  the  correlation  intensities  given  in  Eqs.  (9) 
and  (10),  we  obtain  a  new  value 

h  =/*,(«)  + /w(«)  =  4Cf(0).  (11) 

which  is  independent  of  the  orientation  angle  «.  Therefore, 
we  conclude  that  rotation  invariant  correlation  is  achieved. 
Using  Eqs.  (9)  and  (10),  the  value  of  a  can  I  s  determined  as 

<r  =  |tan (12) 

However,  /*.  nn4  are  positive  defined  quantities;  hence 
the  value  of  «-Qv;  jrmined  Eq.  (12)  is  confined  to  region  |0,|7r/ 
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Fig.  4.  (a)  Plot  of  the  resultant  peak  intensity  vs  the  input  target 
orientation  for  objects  5  (upper  curve}  and  3  (lower  curve),  respec¬ 
tively.  (b>  Plot  of  the  calculated  rotaiiou  angb  »  vs  input  target 
orientation. 


(2fc)]}.  Consequently,  for  each  a  thus  computed,  there  arc  4k 
possible  orientations  of  the  target. 

To  insure  a  non-negative  value,  bias  terms  can  be  added  to 
R*i  and  R«-  The  positive  definite  images  can  then  be 
written  onto  the  input  SLM,  in  which  the  undesired  bias  can 
be  removed  at  the  frequency  plane.  On  the  other  hand,  it  is 
very  difficult  to  implement  a  complex  function  onto  a  SLM. 
However,  in  a  recent  paper  by  Leclerc  et  al.,~  it  is  shown  that 
by  using  a  binary  phase-only  technique,  a  circular  harmonic 
expansion  filter  can  also  be  implemented  directly  on  a  SLM 
in  a  VaiiderLugt  correlator  architecture. 

Figure  2  consists  of  the  target  image  5  located  in  the  lower 
half,  und  the  real  and  imaginary  parts  of  the  second-order 
CHE  of  the  character  are  located  in  the  upper  half.  The 
input  pattern  recorded  on  n  photographic  film  was  then 
inserted  at  the  input  plane  of  the  <ITC  shown  in  Fig.  1,  and 
the  two  correlation  peak  intensities  with  respect  to  the  real 
and  imaginary  CHE  were  recorded  at  the  output  plane.  Fig¬ 
ure  !)  shows  the  output  correlation  result  when  the  turgel  is  at 
0°.  Figure  4(a)  shows  the  sum  of  these  two  intensities  as  a 
function  of  the  target  rotation.  We  can  see  that  the  peak 
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intensity  is  independent  of  the  angular  rotation  of  the  target. 
Therefore,  rotation  invariant  pattern  recognition  is 
achieved.  V  >en  another  numerical  character  3,  which  has 
great  similar. ty  to  the  character  5,  was  used  as  the  target 
image,  the  correlation  response  shows  a  high  degree  of  fluctu¬ 
ation.  This  illustrates  the  discrimination  of  this  filter.  Fur¬ 
thermore,  the  orientation  a  of  the  target  was  also  calculated 
by  using  Eq.  ( 12).  The  plot  of  a  vs  the  target  orientation 
presented  in  Fig.  4(b)  shows  that  the  experimental  result 
obtained  is  within  an  accuracy  of  ±5°. 

It  must  be  mentioned  that  this  technique  requires  summa¬ 
tion  of  two  correlation  peak  intensities  for  each  input  target. 
This  is  a  major  drawback  of  this  technique  and  introduces 
certain  degrees  of  difficulties  in  practice.  One  of  the  two 
peaks  might  fall  below  the  noise  level  of  the  detected  signal 
and  would  be  impossible  to  detect.  However,  the  comple¬ 
mentary  peak  in  this  situation  emerges  to  give  a  high  correla¬ 
tion  peak  intensity  based  on  the  equation  that  sin'J0  +  cos'-’fl  = 

1.  Therefore,  one  of  the  two  peaks  would  be  adequate  to 
detect  the  target  in  this  situation. 

When  two  or  more  targets  are  present  at  the  input  scene, 
multiple  correlation  signals  are  produced.  Therefore,  one 
also  has  to  ensure  that  proper  pairs  of  correlation  peaks  are  to 
be  added.  We  note  that  the  space  invariant  property  of  a 
JTC  provides  one  with  the  position  information  of  the  target 
based  on  the  location  of  the  correlation  peak.  Therefore,  if  a 
peak  is  located  at  position  A  on  the  output  plane,  its  compli¬ 
mentary  peak  must  be  located  at  a  location  dictated  by  the 
value  of  A  and  the  predefined  relative  position  of  the  two 
references.  The  correct  complementary  peak’s  intensity  can 
then  be  obtained  accordingly. 

In  summary:  The  real  value  implementation  of  a  circular 
harmonic  function  in  a  JTC  has  been  investigated.  By  using 
the  real  and  imaginary  parts  of  circular  harmonic  reference 


functions,  rotation  and  shift  invariant  object  detection  can 
be  carried  out  with  a  JTC.  The  advantage  of  this  techni.  ue 
is  that  the  reference  images  are  a  positive  real  value,  which 
can  easily  be  implemented  on  programmable  SLMs. 
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A  study  of  the  effect  of  fringe  binarization  on  the  joint 
transform  correlator  has  shown  that  the  interference 
fringes  for  multiple  targets  could  produce  false  alarms  and 
misses. 

One  of  the  important  features  of  an  optical  correlator  is 
baaed  on  the  shift  invariant  property  of  the  Fourier  transfor¬ 
mation  which  allows  multiple  targets  to  be  detected  and 
tracked.  In  an  optical  joint  transform  correlator  (JTC),1'3 
both  the  reference  object  and  the  input  targets  are  located  in 
the  input  plane  of  the  coherent  optical  processor.  The  corre¬ 
sponding  joirt  transform  power  spectrum  of  the  input  ob¬ 
jects  (i.e.,  the  reference  object  and  input  targets)  consists  of 
various  carrier  interference  fringes,  which  can  be  extracted 
by  a  square  law  device  such  as  an  image  detector.  By  inverse 
Fourier  transforming  these  interference  fringes,  the  cross 
correlations  between  reference  object  and  input  targets  are 
produced. 

In  this  Letter,  the  effects  of  binarization  of  the  power 
spectrum4  in  a  JTC  are  discussed.  We  assume  that  the 
reference  object  f0{x,y)  and  the  input  targets  f,(x,y),  i  =  1, 
2,. . .  are  positioned  at  (a,,?,),  i  -  0,1,. . .  4V  as  shown  in 
Fig.  1.  With  coherent  illumination,  the  complex  light  distri¬ 
bution  at  the  Fourier  plane  Pi  is  given  by 
,v 

Glp,q)  =  y  F,{p,q)  exp [-jlaj)  +  b  7)],  (1) 

1  .  j 

where 

=  5'(/,(x.y)j,  1  =  0,1,2 . (2) 
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Fig.  1.  Joint  transform  optical  correlator  using  binarized  interfer¬ 
ence  fringes. 


1  denotes  the  Fourier  transformation,  and  (p,q)  is  the  angu¬ 
lar  spatial  frequency  ay  y.rr'e  syi  t/n. 

The  corresponding  '.niensitv  ’istr.  bution  captured  by  the 
CCD  camera  can  be  written  as 


where  *  represents  the  complex  conjugate.  The  output  light 
distribution,  through  inverse  Fourier  transformation,  can  be 
shown  to  be 


g(x,y)  -  x-' 

.wt 

.V 

+  V  JU*  “  («o  “  °t)*y  -(b0-b  )q) 

.v 

+  X  +  *a° a»^  +  ^0  “  &*)<?! 

i-i 

.Y-l  N 

+ y  y  &iki*  -  {at  -  o*)*y  -  (6,  -  6*)^] 

i-i  *-i+i 
.V-l  N 

y  ^  +  +  (4) 

1-1  *-1+1 

where  Rib.(x,y)  represents  the  cross  correlation  between 
ft(x,y)  and  /*(x,y).  To  separate  the  cross  correlation  be¬ 
tween  the  reference  object  and  the  targets  [i.e„  R0 ,(x,y)  and 
f?io(*.y)l  from  the  cross  correlation  between  input  targets 
themsf  es,  we  let  the  separation  between  the  reference  ob¬ 
ject  and  input  targets  be 


!Cf‘h>,tf)r  -  V  lF,(p,(j)F 

, 

s 

+  ^  r 
Tf 

X  esp| -j[(a0 -sVt  {*„  -  b.lgjj 
.v 

+  y  *o  (p,q)F,(p.q) 


X  eipi-;[(-a0  +  a.ip  +  (~b0  +  &,)<?]{ 
.v-i  ,v 

+  V  y  F,(p,q)F'k(p,q) 

1-1  *-!+! 

X  exp|-j((a,  -  a„)p  +  (b,  -  bk)q j| 

•V-l  ,v 

+  y  y  K (p,q)Fk(p,q), 

l-l  *«C+t 

X  exp|— ;[(— a,  +  nk)p  +  (- b .  +  b*)?]!. 


min  [(aft  -  a.)2  +  (fc0  -  fc,r) 

.—i.  _v 

>  max  [(a,  —ak)2  +  (b,  -  6*)2].  (5) 
.Ji 

If  the  input  targets  are  identical  to  the  reference  object, 
i-e.,  fi(x,y)  =  f(x,y)  for  all  i,  Eq.  (3)  becomes 

lG(p,<?)l2  =  (IV  +  l)|F(p,o)|2 


IT 

2  ^  lF(p,q)i2  cos((a0  -  a,)p  +  (60  -  6,)qj 
1-1 

N-\  S’ 

2  y  y  \F(p.q)\2 

— W  M 


.-1  A-l+l 

X  cos|(«.  -  ak)p  +  (6,  -  bk)q). 


(6) 


Thus  the  carrier  interference  fringes  can  be  seen  in  the  cosine 
terms.  By  properly  thresholding  Eq.  (6),  the  binarized 
fringe  pattern  can  be  described  as 


(3) 


...  ,  fl,  I(p,q)>l 

(P,0“|°*  /(MX/ 


i'r- 

T’ 


(7) 


Fig.  2.  (a)  Input  plane  geometry.  Fare  used  as  the  reference  object  and  input  targets.  '  b)  Output  correlation  peaks  alter  fringe  binarization. 
(c)  Output  correlation  peaks  for  noisy  targets  in  (a)  after  fringe  binarization. 
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where  Hp,q)  is  the  intensity  distribution,  i.e.,  \G(p,q)\2,  and 
It  the  selected  threshold  level. 

Let  us  first  consider  a  single  input  target  case,  then  the 
binarized  pattern  H(p,q)  would  become  a  rectangle  wave 
function,  which  contains  the  carrier  frequency  (i.e.,  interfer¬ 
ence  fringe  frequency)  and  its  harmonics.  By  inverse  Fouri¬ 
er  transforming  this  function,  a  series  of  correlation  spots 
will  be  produced.  The  first-order  correlation  spots  repre¬ 
sent  a  match  of  the  input  targets  with  respect  to  the  reference 
object.  However,  if  the  input  is  composed  of  multiple  tar¬ 
gets,  the  joint  transform  interference  fringes  will  be  very 
complicated  and  the  binarization  of  these  fringes  could  mask 
the  carrier  frequency  content  and  possibly  generate  unwant¬ 
ed  frequency  components.  Therefore,  binarization  of  a  joint 
power  spectrum  for  multiple  targets  will  generally  cause  false 
alarms  and  incorrect  identification  in  the  output  correlation 
plane. 

Computer  simulatic  s  are  now  given  to  predict  the  effects 
of  fringe  binarization  on  the  JTC.  In  the  first  example, 
shown  in  Fig.  2(a),  the  character  E  at  the  right-hand  side  is 
used  as  the  reference  object  and  the  other  characters  on  the 
left-hand  side  are  used  as  input  targets.  From  this  configu¬ 
ration.  the  correlation  between  the  reference  and  targets 
should  be  located  inside  a  viewing  window  given  by 

3/i  <  x  <  8h  and  —h  <  y  <  h, 
or 

~8h  <  x  <  -Zh  and  —h  <  y  <  h.  (8) 

It  is  assumed  that  the  character  £  occupies  15  X 15  pixels  and 
the  separation  h  is  17  pixels.  A  512  X  512  point  fast  Fourier 
transform  was  used  in  the  calculation.  Figure  2(b)  shows  the 
output  correlation  peaks  within  the  viewing  window.  A  false 
alarm,  due  to  the  binarization  process,  can  be  seen  in  this 
simulation.  In  other  words,  if  the  nonlinear  operation  works 
well,  there  should  be  three  correlation  peaks  in  the  viewing 
window.  In  fact,  as  shown  in  Fig.  2(b),  there  are  four  correla¬ 
tion  peaks,  one  of  which  is  apparently  the  result  cf  the 
binarization  process.  We  further  demonstrate  that,  if  the 
input  targets  [e.g.,  Fig.  2(a)]  are  embedded  in  additive  Gauss¬ 
ian  noise  (with  SNR  =  15  dB),  a  false  alarm  an'*  a  miss  are 
produced,  as  shown  in  the  simulation  result  of  Fig.  2(c). 
Moreover,  if  an  A,  instead  of  the  E,  is  used  as  the  reference 
object,  as  depicted  in  Fig.  3(a),  a  false  alarm  could  also  be 
produced,  as  shown  in  Fig.  3(b),  even  though  the  reference 
and  targets  are  different.  In  this  figure,  we  see  a  strong  peak 
appear  uninvitedly  at  a  WTong  position,  which  is  purely  due 
to  the  binarization  process.  Several  runs  of  simulation,  un¬ 
der  low  level  noise  environment  (e.g.,  SNR  >  15  dB),  have 
been  tested.  The  positions  of  the  false  alarm  and  the  miss 
remain  unaltered.  Mention  should  be  made  that  any  change 
of  the  positions  of  the  false  alarm  and  the  miss  has  not  been 
observed  in  various  runs  of  simulation,  with  different  noise 
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disturbances  of  the  same  level.  Thus  we  see  that  binariza¬ 
tion  of  joint  transform  spectral  fringes  would  generally  intro¬ 
duce  false  alarms  and  misses  for  multitarget  correlation. 

Summarizing:  we  have  examined  the  effects  of  the  fringe 
binarization  on  the  joint  transform  correlator  via  computer 
simulation.  The  simulated  results  show  that  the  binarized 
joint  transform  interference  fringes  for  multiple  targets 
could  produce  false  alarms  and  misses.  Fringe  binarization 
may  not  be  usefully  applied  for  multiple  target  recognition, 
although  it  can  be  applied  successfully  to  noise  free  single 
target  correlation. 

We  acknowledge  the  support  of  the  U.S.  Army  Research 
office  contract  DAALU3-87-0147. 
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Fig.  3.  (a)  Input  plane  geometry.  A  is  used  as  the  reference  object, 
(b)  Output  correlation  peaks  aftc  fringe  binarization. 
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Comparison  of  detection  efficiencies  for  VanderLugt  and  joint 
transform  correlators 
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The  correlation  of  peak  intensities  of  VanderLugt  and  joint  transform  correlators  with  a  single  object, 
multiple  objects,  and  noisy  environments  are  analyzed.  The  study  shows  that  the  VanderLugt  correlator  can 
generally  perform  batter  for  the  multiple  object  case  and  also  under  a  noisy  environment. 


I.  Introduction 

The  VanderLugt  correlator1  (VLC)  and  the  joint 
transform  correlator2  (JTC)  are  the  major  types  of 
optical  correlators  used  for  pattern  recognition  appli¬ 
cations.3"9  In  this  paper,  we  investigate  the  detection 
efficiencies  of  these  two  types  of  correlator  for  multiple 
objects,  and  within  a  noisy  environment. 


|F(0,0)|  HF(/„/y)U.  (3) 

At  the  output  plane,  the  complex  light  distribution  can 
be  written  as 

g(xc,)  =  [|F(0,0)l  +>  mx' y)  * fix' y)  ®  fix,y)  +  A2fixy) 

+  Af(x  +  \ff0j)®f(x  +  \ff0,y) 


li.  Basic  Analysis 

In  the  following,  we  review  the  basic  mathematics 
associated  with  the  VanderLugt  and  Joint  Transform 
correlators  to  establish  notation.  The  complex  spatial 
filter  as  obtained  with  the  VanderLugt  technique  can 
be  written  as 

HQ.  ,fy)  =  \FQ„fy)\2  +  A2  +  2A\F(fJy)\ 


X  cos(2t/qX  -  <t>(fjy)],  (1) 


where  \fx  -  x/(\f),fy  =  y/(X/)]  are  the  spatial  frequency 
coordinates,  fo  is  the  spatial  carrier  frequency  of  the 
matched  filter,  A  is  the  amplitude  of  the  reference 
wave,  X  is  the  illuminating  wavelength,  { is  the  focal 
length  of  the  Fourier  transform  lens,  and  F(fx,fy )  = 
\F(fx>fy)\  exp[ttf>(/x,/y))  is  the  Fourier  transform  of  the 
object  f(x,y). 

The  normalized  amplitude  transmittance  of  the  spa¬ 
tial  filter  can  be  written  as 


H(ft>fr)  =  • 


1 


■l|F(/J./v)|I  +  A2 


1|F(0,0)!  +  A\ 2 
+  2A\F(fx,fy)\  cos(2s/ox  -  oQJy)]\, 

where  we  have  assumed 


(2) 
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+  Af(x-\lf0ly).f(x-\ff0,y)\,  (4) 


where  *  and  ®  represent  the  convolution  and  the  corre¬ 
lation  operation,  respectively.  The  third  term  of  the 
above  equation  represents  the  autocorrelation  of 
f(x,y),  in  which  the  correlation  peak  intensity  is  given 
by 


Iv  =  - 


A2 


[|F(0,0)|+A1* 


|/(^.>)|'  dx  dy 


(5) 


For  a  given  signal,  the  equation  is  a  function  of  the 
reference  beam  amplitude.  The  optimum  correlation 
peak  intensity  occurs  for  unity  reference-to-object 
beam  ratio  {i.e.,  A  =  |F(0,0)|).  The  corresponding 
output  peak  intensity  can  be  written  as 


Iv- 


I6|f  (0,0)| 


;  |  |  J/U,y)|: 


dx  dy 


(6) 


We  shall  now  provide  the  same  calculation  for  a  joint 
transform  correlator.  Let  the  amplitude  transmit¬ 
tance  at  the  input  plane  be 


dx^)  =  fix  -  x„,  y)  +  fix  +  x,„y),  (7) 

where  one  of  the  input  objects  serves  as  the  reference 
image. 

Since  the  square  law  detector  at  the  Fourier  plane 
takes  the  square  modulus  of  the  Fourier  transform  of 
£(x,>'),  the  normalized  amplitude  transmittance  of  the 
JTC  filter  can  be  written  as 


H(f„fs) 


\FiUtf 

2|F(0.0)|3 


+  cost  4  «(/,)]  • 


18) 
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Assuming  unity  illumination  of  this  filter,  the  out¬ 
put  complex  light  distribution  is  given  by 

elw) "  8  /l,:  0,1 

+  fix  +  2x„,y)  ®  fix  +  2x,„y) 

+  fix  -  2x„,y)  ®  fix  -  2xv ,y)],  (9) 


in  which  the  correlation  peak  intensity  at  (2xo,0)  or 
(-2x0, 0)  is 


16|F(0,0)| 

From  Eqs.  (6)  and  (10),  we  have 


(10) 


y  =  |F(0,0)|2.  (11) 


So  far  we  have  assumed  unity  read-out  illumination  for 
both  the  VLC  and  the  JTC.  If  the  read-out  intensity 
for  the  JTC  is  B2  instead  of  unity,  then  Eq.  (11)  would 
be  equal  to  |F(0,0)|  2/£2.  Unless  B  is  equal  to  |F(0,(b] , 
I v  would  not  equal  Ij.  This  means  for  a  smaller  object 
with  lower  average  transmittance,  the  ratio  Iv/Ij  is 
low.  Thus,  the  relative  performance  would  favor  the 
JTC  for  smaller  objects  with  lower  average  transmit¬ 
tance.  An  example  can  be  given  to  demonstrate  this 
effect.  Let  the  average  amplitude  transmittance  of 
the  object  t  and  its  spatial  dimension  be  X2.  Then,  the 
zero-orat-r  Fourier  diffraction  would  be  confined  with¬ 
in  the  region  of  (X//X)2.  Hence,  the  intensity  at  the. 
origin  of  the  Fourier  transform  plane  is  t2X*/(Xf)2  for 
unity  illumination.  To  make  Iv  smaller  than  Ij,  for  X 
=  6  X  10'4  mm  and  f  -  300  mm,  t  should  be  smaller 
than  0.18  for  an  object  size  of  1  mm2. 


IK.  Multiobject  Detection 

For  the  case  of  multiple  objects,  the  amplitude 
transmittance  of  the  VanderLugt  matched  filter,  as 
given  in  Eq.  (2),  is  independent  of  the  input  objects. 
The  input  amplitude  transmittance  is  assumed  to  be: 


M 

t.uv(xy)  -  y7(* - *i  <*,<x3 .  ■■  <xM,  (12) 

where  we  have  M  nonoverlapping  objects  distributed 
on  the  x-axis. 

Thus,  the  output  complex  light  distribution  would 
be 


Rm  <*•>■)  = 


1 

||F(0.0)|  +  A): 


M 

V(/(x 

I 


*„.>•) 


•  fix  -  x_,y)  ®  fix  -  xn,y) 

+  A-fix  -  x.„y)  +  Afix  +  Xff„  -  x,,y) 


©  fix  +  Xff„  -  x.„y) 


+  Afix  -  ,\//„  +  x„,.v)  .  fix  -  \ff„  +  x„,y)|,  ( IS) 

where  the  width  of  each  object  is  assumed  to  be  X.  If 
A//, i  is  greater  than  xm  -  x  i  +  (5/2)X  the  outputs  do  not 
overlap.  The  corresponding  correlation  peak  intensi¬ 
ty  at  i~Xff„  +  x„,0)  is  then  identical  to  that  of  Eq.  (6). 
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For  the  JTC,  the  input  transmittance  can  be  written 

as 


M 

'■mi  =  fix  +  i„.y)  +V/(x-  xny),  i  14) 

n- i 

where  the  first  term  represents  the  reference  object. 

After  square  law  detection,  the  normalized  ampli¬ 
tude  transmittance  of  the  JTC  filter  is 


H.mMzJJ  - 


I  F(U)\2 


(1  +  A/)-|F(0.0>|- 

M 

x  J 1  +  M  +  2  V  cos[2x(x0  +  xn)fx] 

*«  I 


+  2  Y  cos[2jr(xm  -  x, 


n<m 


(15) 


The  corresponding  output  light  distribution  can  be 
shown  to  be 


g.uAx<y)  = - - - 

(1  +  A02|F(0,0)|2’ 


( 1  +  M)fix,y)  ®  /(x,y) 


+  y  fix -x„  +  xn,y)  ®  fix  -x0  +  xny) 

n»l 
M 

+  y  /(*  +  x0  -  *„,y) «  fix  A-  x0  +  xn,y) 

nm\ 

M  » 

+  y  y  fix  -xm  +  Xny)  ®  fix  -  xm  +  Inv>)  ^ .  (16) 

n-1 
n*«m 


Thus,  the  correlation  peak  intensity  at  either  (xo  - 
x„,0)  O’  (~xo  +  xn,0)  can  be  written  as 


l 


(i+Aoimo)!4 


a: 


1  fix,y)\~dxdy 


(17) 


It  is  apparent  that  the  intensity  of  the  correlation 
peak  decreases  very  rapidly  as  the  number  of  the  ob¬ 
jects  M  increases.  The  reason  is  that  the  wave  ampli¬ 
tude  at  the  origin  of  the  Fourier  transform  plane  is 
proportional  to  the  number  of  input  objects,  which 
includes  the  refer  ence  image.  This  is  essentially  limit¬ 
ed  by  the  maximum  amplitude  transmittance  of  the 
JTC  filter,  which  "an  not  exceed  unity.  Furthermore, 
from  Eqs.  (17)  and  (6),  the  correlation  peak  intensity 
ratio  between  the  JTC  and  the  VLC,  for  the  multi¬ 
object  case  can  be  written  as 


= - JS -  (l8) 

/«>  (l  +  Af:lA‘(0.0)|* 

Since  the  output  correlation  intensity  from  the  JTC  is 
dependent  on  the  number  of  input  objects,  Eq.  (18) 
shows  that  the  performance  of  the  VLC  would  general¬ 
ly  be  better  than  the  JTC,  unless  ( 1  +  A/)-|F(0,0)|  <  4. 

Figures  1  (a)-(d)  show  the  cor'e*ation  peak  intensity 
as  a  function  of  the  number  of  input  objects  for  differ¬ 
ent  vulues  of  | /'’«).())(-.  Since  the  A'anderLugt  filter  is 
independent  of  this  input  object  illumination,  the  VLC 
peak  intensity  remains  constant  regardless  of  the  num- 
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F.g.  1.  Correlation  peak  intensity  function  of  the  number  of  objects  M:  iMJ  and  /  w.  peak  intensity  or  .JTC  and  VLC.  respectively;  (a)  for 
|F(0.0)|- -  0.031.3:  (b)  for|F(0,0)|‘  =  0.125;  (c)  for|."(0.0)j2  =  0.5:  (d)  for|F(0.0i|2  =  2.0. 


ber  of  input  objects.  However,  the  peak  intensity 
decreases  very  rapidly  for  the  JTC  as  the  number  of 
input  objects  increases.  Furthermore,  the  relative 
performance  of  the  JTC  and  VLC  depends  on  the 
number  of  input  objects  for  |F(0,0)|  <  1.  This  can  be 
demonstrated  by  looking  at  Fig.  3(a)  in  which  the  JTC 
performs  better  only  when  the  number  of  input  objects 


is  smaller  than  three.  However,  if  |F(0,0)|  >  1,  the 
VLC  would  always  perform  letter  than  the  JTC. 

IV.  Detection  in  a  Noisy  Environment 

We  shall  now  investigate  the  case  for  input  objects 
imbedded  in  a  noisy  environment.  In  this  case,  the 
input  transmittance  function  can  be  written  as 
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M 

tsuAxc/)  =  V  /(*  -  xncr)  +  n(xO'),  (i9) 

n“l 

where  n(x,y)  represents  additive  noise.  Since  the  am¬ 
plitude  transmittance  of  a  VanderLugt  filter  is  inde¬ 
pendent  of  the  input  object,  the  complex  light  distribu¬ 
tion  at  the  output  plane  can  be  shown  to  be 


3stv(*<y)  - 


[|F(0,0)|+A]2 


\}(x  -  xny)  *iix-  x„ O') 


Qfl*~x„j) 


+  A2f(x  -  x„y)  +  Afix  +  X//0  -  xnly) 


The  output  correlation  peak  and  noise  intensity  can  be 
shown  to  be,  respectively, 


lU  +  M)|F(o.o)|+|mo)|)< 

x  ||  ^'\f{xj)\2dxdy  2.  (25) 

_ 1 _ 

[(l  +  AOiF(0,0)|+|N(0,0)|l< 

xlff  nixy)fixy) dx  dy\  ■  (26) 


«  /(x  +  Mfo  ~  xny) 

+  Af(x  -  X//0  +  xnj)  •  f(x  -  Xff0  +  xnly)J 
+  n{xy)  •  f(x,y)  ®  f{x,y)  +  Ahi'xy) 

+  Anix  +  \ffoj)  ®  fix  +  X;V'j  -  xny) 

+  Anix  -  \fi0,y)  •  /(x  -  X//a  +  x„,y)|  j-  (20) 

Thus,  the  output  correlation  peak  intensity  is  un¬ 
changed  from  that  given  in  Eq.  (6).  The  output  noise 
intensity  can  be  written  as 


r”"  ■  ism#  |/  C  •  <*» 

The  output  signal-to-noise  ratio  can  be  expressed  as 


(1)  .-Ui 

Wvlc  If  r* 


f  j*jf(xj)l!dxdy 


nfx.yjpixj)  dx  dy 


For  a  noisy  input  in  a  JTC,  the  input  is  given  by 

M 

1m(xj)  -  fix  -  ZoO?)  +  /(x  -  xny)  +  n(xj').  (23) 

n"l 

The  amplitude  transmittance  of  the  JTC  filter  can  be 
written  as 


XuMM  - 


[«  +  M)|F(0,0>|  +|iV(0.0)|) 


?  I  F(AJ 


J,)|sj  1  +  Af 


+  2  y  cos[2x(x0  -  xn)ft] 


+  2  Y  y  co*[2r(x_  -  x.);,j 


+  FifMS‘<fJ>)  exp[-<‘2xx^,J 
+  Fmif,Jy)S’(f,Jy)  exp[i2-x,/J 

Af 

+  F<fJs)N‘<fJA  V  eipKaxi./.l 


+  F*(/,4)-W,/.) 

X  V  exp[<2xz„/t|  +|iV(/,4)!:T 


From  these  results,  it  is  observed  that  the  output  S/ 
IV  is  the  same  for  both  the  JTC  and  the  VLC.  Howev¬ 
er,  the  correlation  peak  intensities  are  quite  different 
For  the  JTC,  the  correlation  intensity  is  dependent  on 
the  presence  of  input  noise,  while  for  the  VLC,  the 
peak  intensity  is  independent  The  ratio  of  the  peak 
intensity  between  these  correlations  can  be  written  as: 

16|F(0,0)|2 

V  1(1  +  .M)jF(0,0)l  +  |.V(0.0)|14 ' ’* 

Figure  2(a)  shows  logarithmic  plots  of  as  a 

function  of  Af,  for  various  noise  levels  |1V(0,0)|2/ 
|F(0,0)|2.  For  a  certain  noise  level,  ln(tMlTm)  de¬ 
creases  monotonically  as  Af  increases.  On  the  other 
hand,  for  a  fixed  number  of  input  objects,  the  value  of 
IntfMjfffiiv)  becomes  smaller  as  the  noise  level  in¬ 
creases.  In  the  region  above  the  Af-axis,  the  correla 
tion  peak  intensity  for  the  JTC  is  higher  than  that  for 
the  VLC.  In  other  words,  the  JTC  would  perform 
better  in  this  region.  On  the  other  hand,  the  VLC 
would  perform  better  than  the  JTC  in  the  region  below 
the  Af-axis.  Furthermore,  the  curves  in  Fig.  2{a)  are 
shifted  down  as  |F(0,0) j2  increased.  This  implies  that 
due  to  the  limited  dynamic  range  of  the  square  law 
converter,  the  JTC  correlation  intensity  decreases  if 
the  average  transmittance  function  of  the  object  be¬ 
comes  larger.  Logarithmic  correlation-peak-intensity 
ratios  as  a  function  of  noise  level  for  various  number  of 
input  objects  are  plotted  in  Fig.  3.  It  is  apparent  that 
II mv )  decreases  monotonically  as  |Ar(0,0)|2/ 
|F(0,0)|2  increases.  With  the  same  noise  level,  the 
value  of  the  correlation  intensity  ratio  becomes  lower 
as  the  number  of  input  objects  increases. 


V.  Conclusion 

We  have  studied  the  detection  efficiencies  for  the 
uTC  and  the  VLC,  with  multiobject  scenes  and  in  a 
noisy  environment.  For  the  single-object  case,  it  was 
found  that  the  relative  performance,  between  these 
two  correlators,  depends  upon  the  object  transmit¬ 
tance  function  and  the  object  size.  In  general,  the 
VLC  would  perform  better  unless  the  input  object  is 
very  small  with  very  low  average  transmittance. 

For  the  multi-object  case,  the  JTC  peak  intensity 
decreases  rapidly  as  the  number  of  input  objects  in¬ 
creases.  Since  the  VanderLugt  filter  is  independent  of 
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input  noise,  the  VLC  peak  intensity  remains  constant 
regardless  the  number  of  input  objects.  When  jF(0,0)| 
<  1,  the  relative  performance  depends  on  the  number 
of  input  objects.  The  JTC  may  perform  better  if  (1  + 
1W)2|F(0,0)|  <  4.  However,  for  |F(0,0)|2  >  1,  the  VLC 
would  always  perform  better  than  the  JTC. 

Although  the  output  signal-to-noise  ratios  for  both 
correlators  are  the  same,  their  correlation  peak  intensi¬ 
ties  are  quite  different.  Since  the  VanderLugt  filter  is 
not  changed  by  input  noise,  it  should  generally  per¬ 
form  better  in  a  noisy  environment,  except  for 
|m0)|  [1  +  M  +  |1V(0,0)|/|F(0,0)|]2  <  4. 

We  finally  remark  that,  in  terms  of  detection  effi¬ 
ciency,  the  VLC  will  generally  perform  better  when 
compared  to  the  JTC.  However,  with  respect  to  real¬ 
time  applications,  the  JTC  still  offers  the  advantages 
in  spatial  filter  synthesis,  alignment  of  spatial  filter, 
and  overall  simplicity. 
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Adaptive  joint  transform 
correlator  for  real-time  colour 
pattern  recognition 

F.T.S.  YU,  S.  JUTAMULIA*,  R.V.  YELAMARTY,  D.  A.  GREGORY 

Colour  liquid  crystal  televisions  (LCTVs)  can  be  used  to  perform  real-time  multi¬ 
channel  joint  transform  correlation  capable  of  discriminating  between  different 
colours  and  different  shapes.  In  the  future  this  optical  system  may  be  combined 
with  a  digital  knowledge-based  inference  processor.  Preliminary  optical 
experimental  results  are  presented,  which  demonstrate  how  such  a  system  would 
work. 

KEYWORDS:  liquid  crystal  televisions,  colour  recognition,  shape  recognition,  joint  transform 
correlation 


Introduction 

Recently,  we  have  demonstrated  real-time  polychromatic 
pattern  recognition  using  a  commercially  available 
inexpensive  colour  liquid  crystal  television  (LCTV)  and 
mu'tiple  Vander  Lugt  matched  spatial  filters1.  This 
method  has  the  important  limitation  of  changing  the 
reference  pattern.  To  improve  the  adaptability  of  the 
colour  pattern  recognition  system,  a  joint  transform 
correlation  processor  which  has  been  successfully  applied 
to  monochromatic  pattern  recognition2,  is  proposed. 

The  reference  colour  pattern  then  may  simply  be 
generated  by  a  computer  or  a  CCD  camera. 

The  uses  of  LCTVs  in  optical  processing  and 
computing  have  been  widely  reported*"1*. 
Monochromatic  real-time  pattern  recognition  based  on 
Vander  Lugt  matched  spatial  filtering  techniques  was 
demonstrated  by  Liu  et  al4  and  Gregory5.  A  non¬ 
coherent  pattern  recognition  technique  using  LCTVs 
was  reported  by  Jutamulia  et  al14.  which  can  be  directly 
extended  to  colour  object  identification.  Colour  pattern 
recognition  has  also  been  performed  using  a  colour 
sensitive  matched  spatial  filter1 51 7,  a  deflecting 
prism18,  and  a  diffraction  grating19"21.  In  this  paper, 
a  technique  which  uses  colour  LCTVs  in  the 
construction  of  a  multi-channel  joint  transform 
correlator  will  be  described.  This  system  could  be 
considered  as  a  model  for  future,  optical,  knowledge- 
based  pattern  recognition  systems22. 

Concept 

The  joint  transform  processor  is  an  alternative 
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approach  to  optical  pattern  recognition23'24,  and  has 
proven  to  be  suitable  for  adaptive  programmable 
correlation  because  no  matched  filter  is  required.  In 
general,  it  is  very  difficult  to  generate  a  dynamic 
Vander  Lugt  matched  spatial  filter  in  the  Fourier  plane. 

A  commercially  available  colour  LCTV  (Radio  Shack. 
Catalogue  Number  16-154)  has  been  used  to  construct 
the  multi-channel  joint  transform  correlator.  The 
structure  of  the  display  panel  is  depicted  in  Fig.  la. 
From  this  a  diffraction  pattern  with  well  separated 
colour  spectra  should  be  expected  as  shown  in  Fig.  lb. 
Fig.  2  shows  a  schematic  diagram  of  the  experimental 
arrangement.  The  lower  optical  set-up  in  Fig.  2 
performs  the  joint  Fourier  transform  of  the  reference 
and  test  object  displayed  on  LCTV1  and  LCTV2 
respectively.  The  upper  one  performs  another  Fourier 
transform  operation  on  the  joint  transform  power 
spectrum  to  obtain  the  correlation  signal.  L,  and  L, 
are  lenses  for  collimating  the  beams,  L3  is  a  Fourier 
transform  lens  and  L*  is  a  microscope  objective. 

Basically  this  implementation  utilizes  three  colour 
LCTVs.  illuminated  by  collimated  red  and  green  laser 
light  from  HeNe  and  Ar  icn  lasers  as  shown  in  Fig. 

2.  Ideally,  three  primary  colours  should  be  used. 
Vidicon3  is  a  TV  camera  directed  toward  a  scene  which 
may  or  may  not  contain  the  desired  colour  object.  This 
is  displayed  on  LCTV1  without  passing  through  the 
microcomputer  or  other  digital  system  preserving  the 
high  speed  processing  capability.  LCTV2  is  driven  by 
a  computer  in  which  a  set  of  reference  colour  patterns 
have  been  stored.  The  reference  pattern  displayed  on 
LCTV2  can  be  chosen  directly  by  an  operator  or 
determined  by  software  control  of  the  feedback  input 
from  Vidicon2. 

LCTV1  and  LCTV2  are  arranged  side  by  side  in  the 
same  plane  as  shown  in  the  lower  set-up  of  Fig.  2.  The 
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Fig.  1  (a)  Structure  of  the  colour  LCTV  display  panel:  (b)  schematic 
diagram  of  the  polychromatic  spectral  distribution  at  the  Fourier 
transform  plane  (zero  and  first  order  maxima) 


joint  transform  is  performed  by  the  lower  lens  L3.  The 
pixel  structure  of  the  LCTV  produces  multiple  colour 
spectra  in  the  Fourier  plane  as  shown  schematically  in 
Fig.  lb.  A  microscope  objective  lens  L*  with  a  small 
aperture  selects  and  magnifies  only  one  diffraction 
order  consisting  of  well  separated  colour  spectra.  The 
intensity  joint  transform  distribution  of  the  selected 
diffraction  order  can  then  be  recorded  by  Vidiconl 
which  is  directly  connected  to  LCTV3. 

The  correlation  signal  is  obtained  by  performing  a 
Fourier  transform  of  the  intensity  pattern  of  the  colour 
joint  transform  spectra  displayed  on  LCTV3.  This  is 
done  using  lens  !.,  lens  l,4  is  another  microscope 
ob|cclive  which  selects  and  magnifies  the  colo  ir 
correlation  signal  lor  detection  by  Vidicon2.  If  the  zero 
order  is  selected,  colour  correlation  peaks  will 


Fig.  2  Conceptual  architecture  of  an  adaptive  joint  transform 
correlator 
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Fig.  3  Schematic  diagram  of  a  knowledge-based  real-time 
polychromatic  correlator 

superimpose.  Thus,  a  correlation  peak  which  has  the 
same  colour  as  the  object  (if  three  primary  colours  are 
used)  can  be  observed.  In  approaching  a  knowledge- 
based  correlation  system22,  a  diffraction  order  that 
gives  a  spatially  separated  primary  colour  correlation 
pattern  should  be  selected.  A  computer  would  analyse 
the  independent  colour  signals  and  then  determine  the 
colour  of  the  object.  This  may  require  updating  the 
reference  pattern  with  patterns  from  the  knowledge 
base  for  obtaining  the  best  match.  A  schematic  diagram 
of  this  process  is  depicted  in  Fig.  3. 


Experiments 

To  verify  operation  of  this  knowiedge-based  colour 
correlator  and  the  usefulness  of  the  colour  LCTVs,  a 
preliminary  experiment  has  been  performed.  A 
schematic  diagram  of  the  experimental  set-up  is  shown 
in  Fig.  4.  A  microcomputer  was  used  to  generate  a 
colour  object  and  reference  patterns  simultaneously  on 
a  single  LCTV.  A  black  and  white  picture  of  the 
generated  colour  patterns  (simulated  fighters)  taken 
from  a  colour  video  monitor  is  given  in  Fig.  5a.  Fig. 

5b  shows  the  picture  as  taken  directly  from  the  LCTV 
and  as  can  be  seen,  the  LCTV  has  low  resolution  and 
give  a  colour  ghost  image.  The  colour  ghost  image  is 
due  to  cross-talk  between  neighbouring  colour  pixels. 
For  simplicity,  only  two  primary  colours  (red  and 
green)  were  used.  A  coliimatcd  bichromatic  coherent 
beam  from  HcNc  and  Ar  ton  lasers  is  incident  on  the 
LCTV.  which  was  disassembled  and  immersed  m  a 
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Fig.  4  Experimental  set-up  of  the  adaptive  joint  transform  correli.or 
for  real-time  colour  pattern  recognition 


liquid  gate.  The  polarizers  of  the  LCTV  were  removed 
and  good  quality  polarizers  were  used  in  the  set-up 
shown  in  Fig.  4.  The  joint  transform  of  the  object  and 
the  reference  patterns  is  performed  by  lens  L3.  If  the 
object  and  reference  are  identical  (same  shape  and 
colour),  the  joint  transform  spectrum  consists  of  a  two- 
colour  fringe  structure. 

The  pixel  structure  of  the  display  panel  diffracted  the 
joint  transform  spectrum  into  many  orders.  The  red 
and  green  spectra  were  overlapping  at  the  zero  order. 
However,  they  were  separated  further  at  higher 
diffraction  orders.  The  second  order  of  the  joint 
transform  spectrum  was  selected  because  the  red  and 
green  spectra  were  sufficiently  separated  but  they  were 
still  covered  and  could  be  magnified  by  a  single 
microscope  objective  L*.  These  magnified  independent 
red  and  green  spectra  were  detected  by  a  TV  camera 
and  recorded  by  a  video  cassette  recorder  (VCR).  The 
joint  transform  spectra  displayed  on  a  video  monitor 
are  shown  in  Fig.  6a  for  the  input  of  Fig.  5b.  Fig.  6a 
shows  a  black  and  white  picture  of  green  fringes  on 
the  left  and  red  fringes  on  the  right.  The  green  fringes 
seem  to  be  stronger  than  the  red.  This  might  be  caused 
by  the  larger  green  area  than  the  red  as  shown  in  Figs. 
5a  and  b.  When  one  of  the  simulated  fighters  was 
covered,  the  fringes  disappeared  as  illustrated  in  Fig. 
6b.  The  red  spectrum  was  under-exposed  thus  it  is 
hardly  seen  in  Fig.  6b. 

The  recorded  fringe  structure  was  then  replayed  using 
the  same  LCTV.  To  reduce  colour  cross-talk 
(transmittance  of  red  light  through  green  pixels  and 
vice  versa),  additional  red  and  green  filters  were  used 
to  cover  red  and  green  fringes  of  the  LCTV.  The 
correlation  signals  were  then  input  to  the  computer  for 
the  inference  process.  In  this  preliminary 
demonstration,  a  digital  inference  process  was  not 
attempted.  The  correlation  outputs  were  obtained  by 
taking  the  Fourier  transform  of  the  fringe  structure 
shown  in  Fig.  6a  Half  of  the  magnified  bichromatic 
correlation  output  is  depicted  in  Fig.  7a.  The  extended 
dc  patterns  are  shown  in  the  upper  part  of  Fig.  "a. 

The  dc  patterns  were  due  to  the  background 
transmittance  of  the  low  contrast  LCTV  screen.  flic 


Fig  5  Object  and  reference  patterns  generated  by  microcomputer 
(Apple  II)’  (a)  displayed  on  ordinary  colour  TV  montior.  and  (b) 
displayed  on  colour  LCTV.  The  centre  is  red  and  the  edge  is  green 

Fourier  transform  of  Fig.  6b  is  depicted  in  Fig.  7b. 

The  subtraction  of  the  dc  struture  shown  in  Fig.  7b 
from  Fig.  7a  will  produce  the  correlation  spot  uionc. 
The  subtraction  may  be  included  in  the  digital  inference 
process. 


Concluding  remarks 

flic  application  of  a  colour  LCTV  to  polychromatic 
pattern  recognition  has  been  demonstrated  using  a 
multi-channel  |omt  transform  correlator.  This  should 
he  suitable  for  extension  to  a  knowledge-based  object 
identifier  because  generation  of  the  colour  reference 
p.itlctn  m  i he  lomt  transform  correlator  is  easier  than 


Pig.  S  f  i)  Jqim  transform  power  specM  of  Fig.  5.  the  left  and 
the  right  hinge  structures  are  green  and  red  resoecm/etv:  ;b)  fringe 
structures  disappear  when  one  fighter  m  Fig.  5  is  covered  (the  red 
spectrum  was  underexposed) 


0\ 

fig.  7  la)  Fourier  transform  of  fsg.  6  (a).  The  smalt  spots  at  the 
bottom  indicate  the  corretafton  terms;  (b)  fourier  transform  of  #  Saris 
screen  displayed  e«  m#  LCTV.  showing  the  dc  terms.  The 
subtraction  of  lb)  from  (at  w>it  give  the  correfatmti  terms  alone. 

The  left  and  the  »  qnt  structures  are  green  and  red  respectively 


that  of  the  traditional  Vander  Luet  eorreiator.  The 
system  is  capable  of  discriminating  between  different 
colours  and  different  shapes.  Although  LCTV  possesses 
limited  space-bandwidth  product  (SBP).  it  is  a  low-cost, 
computer-addressable,  spatial  light  modulator  capable 
of  processing  polychromatic  signals.  With  further 
improvements,  the  colour  LCTV  may  be  applicable  to 
colour-senstttve  knowledge-based  paiiem  recognition. 
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Abstract  A  generalized  optical  correlator  is  used  to  analyze  the  performances  of  the 
Vander  Lugt  and  the  joint  transform  architectures.  Noise  performance  and  the  effects 
caused  by  the  differences  between  the  input  and  the  reference  functions  are  analyzed.  It  is 
found  that  the  modulation  index  of  the  joint  transform  filter  decreases  due  to  the  presence 
of  noise  as  well  as  the  differences  between  the  input  and  the  reference  functions.  However, 
it  is  found  that,  in  general,  the  joint  transform  correlator  would  perform  better  than  the 
Vander  Lugt  correlator  and  certainly  has  the  advantage  for  real-time  implementation. 

PACS:  42.30.  42.80 


Two  commonly  used  techniques  for  optical  pattern 
recognition  are  the  Vander  Lugt  [1]  and  the  joint 
transform  [2. 3]  correlators.  With  the  Vander  Lugt  cor¬ 
relator,  investigators  have  used  a  variety  of  techniques 
to  synthesize  multiplexing  matched  filters,  such  as  the 
frequency  [4],  the  angular  [5],  the  color  multiplex¬ 
ing  [6],  the  holographic  lens  array  [7]  and  the  rotating 
grating  technique  [8].  However,  the  joint  transform 
architecture  combined  with  a  TV  camera  such  as 
vidicon  and  a  CCD  can  be  operated  in  a  real-time 
mode  with  ease  [9, 10].  Nevertheless,  the  application 
of  spatial  light  modulators  to  both  Vander  Lugt  and 
joint  transform  correlators  provide  significant  im¬ 
provement  of  the  system  operation  [8-13].  We  shall, 
in  this  paper,  analyze  the  basic  performance  of  the 
Vander  Lugt  (VLC)  and  the  joint  transform  (JTC) 
correlators.  In  the  following,  we  first  present  a  gen¬ 
eralized  optical  correlator  for  both  architectures.  The 
performance  and  limitations  of  these  correlators  are 
evaluated.  Multiplicative  noise  introduced  by  the 
spatial  light  modulators  has  been  used  for  the  analyses. 
Tne  effects  of  the  noise  perturbation  and  the  differences 
between  the  input  and  the  reference  functions  on 
modulation  index  of  the  correlation  filter  arc  cai- 
cu,  '-d.  It  is  shown  that  the  JTC  can  generally  perform 
better  than  the  VLC.  particularly  for  real-time 
implementation. 


1.  A  Generalized  Optical  Correlator 

A  generalized  optical  correlator  can  be  described  by 
dividing  the  system  performance  into  two  parts: 
namely,  the  correlation  filter  construction  and  the 
correlation  operation.  Notice  that  the  term,  “correla¬ 
tion  filter,”  used  here  is  for  both  the  matched  filter  (MF) 
and  the  joint  transform  hologram  (JTH).  The  construc¬ 
tion  of  these  filters  is  basically  using  the  same  proce¬ 
dure  as  shown  in  Fig.  1.  where  an  input  transparency 
h(x.y-b)  and  a  reference  function  gfx.y-r b)  at  the 
input  plane  P,  are  illuminated  by  a  collimated  coher¬ 
ent  light  beam  with  wavelength  A.  The  intensity- 
distribution  of  the  joint  Fourier  spectrum  at  Pz  is  given 
by 

Hu.  r)  =  j/fj2  -r  [G|-  4-  HG’c ' -  //*Ge!isrt .  ( 1 ) 


y.  y* 
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device  can  be  represented  bv 


h(x.y)=  £ 

m.n 


fx  —  md\  /  v  —  nd\ 
x  reel  (  — y—  1  recE  ( ' — '[ —  !  ■  ( 1 0) 

where  Am_n  is  the  |0  or  1)  binary  amplitude  trans¬ 
mittance.  |.V„.nie*2x,>"-“  is  the  multiplicative  noise  in¬ 
troduced  by  the  device.  I  is  the  pixel  size,  d  is  the  penod 
of  the  discrete  structure  of  the  device. 

We  note  that  the  amplitude  noise  can  be  expressed 
in  an  optical  sense  as 

<n> 

where  follows  a  certain  probability  distribution, 

t/Vm.nI=  1  '2.  The  average  value  of  amplitude  noise  is 
K 

AN  s=  —  — .  KS1.  Sm  r  has  random  values  between 

A.  T*  i 

-  J—  and  ■~r-  The  maximum  variation  of  ]:V_  is 
2k  2  K 

.  K- 1 .  . 

from - to  i.  wnten  snows  tnc  nonunnormitv  of  the 

K- f  1 

transmittance  of  the  device.  As  an  example,  if  the 
uniformity  of  the  transmittance  of  the  device  is  80%. 
then  the  average  value  of  amplitude  noise  is  AN  =  4/5. 

For  phase  noise.  Pm-K  represents  a  departure  of  the 
thickness  of  the  (m.  n)  cell  from  the  average  thickness  of 
MOD.  Wc  assume  that  Pmx  follows  a  certain 
ptocability  distribution  and  takes  or.  values  between 
- 1  2-"  and  1  IK.  e.g.,  the  random  value  range  is 
AP  =  1  'K  for  K  ^  1.  For  example,  if  the  surface  of  the 
device  has  •.  flatness  of  /.  4.  then  AP=  '.4.  The  MOD  is 
used  as  an  example  for  noise  model  in  a  real-time 
correlator  Bu  ■'  the  quantity  /  (or  d)  represents  the 
minimum  rcsolv^Se  ceil  of  an  input,  the  above  analy¬ 
sis  can  be  applied  :  evaluate  noise  in  any  optical 
system. 


It  is  trivial  that  if  the  input  object  and  the  reference 
function  are  identical  i.e..  H(u.v)  =  G(u.ui  the  contrast 
of  the  correlation  filter  reduces  to  unity  and  the 
irradiance  of  (12)  becomes  a  sinusoidal  grating  func¬ 
tion.  The  diffraction  efficiency  of  the  grating  is  of 
course  proportional  to  its  contrast  ytOgy^l).  Thus, 
an  ideai  interference  fringe  pattern  with  the  highest 
contrast  is  required  to  obtain  the  best  correlation 
result,  i.e..  the  highest  correlation  peak.  However,  in 
practice,  the  input  object  can  never  be  strictly  the  same 
as  the  reference  function  because  of  noise  disturbances 
in  the  devices. 

If  we  further  consider  the  scale  cha'nge  and  the 
rotation,  the  contrast  y  would  certainly  affect  these 
factors.  Assuming  the  noise  exists  in  the  input  object, 
but  not  in  the  reference  function,  then  y  should  be 
affected  by  the  amplitude  noise  AN  as  well  as  the  phase 
noise  AP.  Since  a  correlation  filter  works  only  in  a 
certain  spatial  frequency  band  ( —  u^,  uj,  ( — t'J,  we 
shall  integrate  the  contrast  y  over  this  band,  and  the 
average  contrast  y  of  the  correlation  filter  over  the 
given  frequency  band  can  be  expressed  as. 


1  t  21HHG1 
4umvm  -L  -tjffi2  +  |G|2 


dudv. 


(14) 


4.  Noise  Effects  on  Contrast  Reduction 

Referring  to  the  correlation  filter  synthesis  of  Fig.  1. 
two  MODs  are  used  as  input  transducers.  One  device 
is  assumed  to  be  contaminated  with  noise  and  the 
another  is  considered  to  be  without  noise.  Since  the 
MOD  is  a  binary  transmission  device,  each  pixel  can 
be  assumed  to  have  a  uniform  transmittance  of 
amplitude  and  phase.  From  (10).  the  amplitude  trans¬ 
mittance  kix.y)  with  noise  and  the  amplitude  transmit¬ 
tance  gix.  y)  without  noise  can  be  expressed  in  terms  of 
the  Fourier  transforms,  i.e.. 


3.  Contrast  of  Correlation  Filter 

Tnc  intensity  distribution  of  the  correlation  filter  Hu.  r) 
of  ( 1 1  can  be  rewritten  as 


H(u.  rl=  v  H.4_  ,:.v_  ' 


x  sinef/u)  sinci/r)e  * 


l(u.v)-'-Hr -iG'r- 2!//i:6'cos[4nbr-Ho— .,»)].  1 12) 

where  //fti.rJ  and  G(u.r)  are  the  Fourier  spectra  of 
/it  v.  vland  gt.x.  yl.  respectively .  Then,  the  contrast  of  the 
interference  fringe  pattern  may  be  represented  by 


G(-  i=  T  f;-4w.«sinc(fa)sinct/f)c‘,2*"5‘**o4r,e*’:str. 

si.B 

f  16) 

where  S\SictZ:r-  *  represents  the  noise.  .4„  „  has  the 
binary  values  J  or  0.  and  2b  deni  ics  the  separation  of 
the  two  devices  in  the  input  plane.  For  simplicity,  only 
the  1-D  case  is  considered.  From  1 1 ).  the  intensity 
distribution  along  the  r  axis  i u  =  0)  in  the  recording 
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plane  can  be  expressed  by 
l  =  /4  sinc2(/t>)  f  Ar  cos  2nndvy 
~  (YA„cos2xndv''j2 
4-  (T  AJNj  cos  2  z(Pn  -  ndv))2 
+  (TAK\NK\sin2-(Pn-ndv)y 

\  r.  j 

4-2  y  An\N„\A'„ cos2n[Pr—(n—n')dv-~2bcY\. 

ff.it'  I 

"  (17) 

Let  us  first  consider  the  amplitude  noise.  i.e~  P„ =0. 
for  all  n=  ±  1,  ±2, ....  Assuming  that  An=  1  for  all 
n=  ±1,  ±2,....  equation  (17)  can  be  reduced  to 

I  =  I4  sinc(lc)2  |YT  cos2zndv)2  +  (Y\NJ,  cos2xndvy 

A  fV|ArJsin2-«diA2 
v*  / 

+  2Ycos2rjidvY\NK\  cos2Mndv—2bv)  j .  (18) 

If I  AM  =  1,  the  intensity  of  (IS)  is  reduced  to 
/  =  2/*  sine2  ((c)  lY,  cos  2n/t  dr'j2  ( 1 4-cos4irbo).  (19) 

and  the  contrast  of  the  filter  (without  noise)  for  the 
above  equation  is  equal  to  1.  Equation  (18)  is  used  to 
calculate  the  intensity  distribution  of  the  interference 
fringes  of  a  correlation  filter,  without  amplitude  noise 

IN  j  =  1 .  and  with  noise,  jN,]  =  ^  -  Nf. ),  where 

i  .  A  4- 1  \a.  / 

K  ^  1.  It  is  assumed  that  A,  is  a  random  variable  taken 
from  a  range  between  - 1/2 K  and  1/2 K.  We  further 
assume  that  N.  obeys  one  of  the  probability  distribu¬ 
tions  such  as  Gaussian,  uniform  or  exponential. 

If  the  noise  introduced  by  the  device  is  only  phase 
noise,  i.c-  |NJ  =  1  for  ail  n=  ±  1,  ±  2. ... .  and  assuming 
that  A,=  I  for  all  n.  then  (18)  becomes 

/  =  "r  sinc:(/t)f  /£eos2iriidFj- 

4- 1 7 cos 2sf P. ~ ndi:)'jz  4-  (T sin2a(P, -ndcYY 

4-  2  y  cos  2  nndv  Y  cos  2m  Pm  -  n  dv  -f  2brj] .  (20) 

*  « 

where  P,  is  a  random  variable  taken  from  u  range 
between  —  i  2 K  and  1  2 K.  which  may  also  obey  some 
probability  distribution. 

Three  typical  noise  statistics  arc  considered:  Gaus¬ 
sian.  uniform,  and  exponential.  In  each  case,  the 


correlation  peak  intensity  is  compared  with  the  case 
without  phase  noise. 

As  an  example,  assuming  that  N„  and  P„  follow  a 
truncated  Gaussian  density  distribution. 


rect 


[W 

L  J  y2.ua 


Is1 


(21) 


It  is  assumed  that  a  =  0  and  that  if  N„  and  P„  take 
values  between  - 1/2 K  and  1/2 K,  then  we  have 


Kgl. 


(22) 


For  simplicity,  the  input  and  reference  functions 
are  1-D  arrays  of  dimension  2 L.  Since  the  spectral 
distribution  of  a  rectangular  function  consists  of 
infinite  lobes  of  finite  bandwidth  1/2  L,  it  can  be  treated 
as  a  periodic  function  in  the  spatiai  frequency  domain. 

It  can  be  seen  that  the  contrast  of  the  filter  is 
reduced  by  the  amplitude  noise,  while  for  the  phase 
noise,  contrast  is  reduced  and  the  fringes  of  the  filter 
are  shifted.  If  the  average  noise  AN  decreases  from 
1/2  to  1/2  K.  where  AfSl,  and  obeys  a  proba¬ 
bility  distribution,  such  as  Gaussian,  uniform,  or 
exponential,  the  average  contrast  is  a  function  of  the 
average  amplitude  noise  as  shown  in  Fig.  3.  The  curve 
showing  contrast  reduction  due  to  Gaussian  ampli¬ 
tude  noise  seems  to  be  a  reasonable  description  of 
amplitude  noise  introduced  by  nonuniform  transmit¬ 
tance  of  tile  device.  When  the  average  value  AN  £0.24 
(Le..  uniform  transmittance  equals  76%),  the  contrast 
of  the  correlation  filter  drops  10%  from  unity.  Figure  4 
shows  the  contrast  as  a  function  of  the  random  value 
range  AP  for  various  types  of  statistical  phase  noise 
(e.g.,  Gaussian,  uniform,  and  exponential  distribu- 


A Vaiu«  aM 

Fig.  J.  Average  contrail  of  ihe  correlaf-m  niter  as  a  function  of 
the  average  amplitude  noise  .I-V  lor  Gaussian  i — i.  uniform 
I  i.  and  evponenfia!  I-  -  •(  amplitude  noise  models 


Companion  of  Vandcr  Lugt  and  Joint  Transform  Correlators 


Assuming  that  the  scale  of  the  input  function  is 
changed,  as  can  be  exnressed  bv 


h(x.  v) 


(Sx\  fSiy+b)\ 

=  rect  |  — )  rect  [  — j  v0)c— 


where  S  is  the  scaie-c'nange  factor,  and  H0  is  a 
weighting  factor,  then  the  corresponding  distribution 
at  the  Fourier  plane  is  given  by 


Hiu.  i  t  =  ihpJn.  v0)  sine  i  2L.  [  ^  —  u0  j  I 

J  I  /  1 


Random  Value  Range  aP 

Rg.4.  Average  contrast  of  the  correlation,  filter  as  a  function  of 
the  random  value  range  JP.  for  Gaussian  ‘ — t  uniform  i — i 
and  exponential  t-  -  -)  phase  noise  models 


By  referring  to  (13).  the  contrast  of  the  correlation 
filter  (assuming  that  H0  =  G0).  can  be  written  as 


2  sine [2 LJiu  -  u0)]  sine  j  2L.  ( ^  -  u0j  sine  [2 L^v  -  r0)]  sine  2L,  0  -  c0  j 


_  1  .  ,  (u  \ 

S1  sine*  f2LJu — «0)]  sine*  [2LJ  r  -  r0)]  +  -3  sine-1  2 Lx  j  -  —  j 


sine2  2LJ  r  -Co  1 ! 


tionsi.  If  JP=l/2.  i.e,  the  surface  roughness  of  the 
devices  is  equal  to  or  smaller  than  }JL  the  contrast  of 
the  filter  is  about  0.9.  for  Gaussian  noise.  From  Figs.  3 
and  4.  we  have  seen  that  a  contrast  of  0.9  yields 
JA'=0.24  for  amplitude  noise  and  dP=  1.2.  for  phase 
noise.  This  means  that  the  surface  quality  of  /.2  causes 
the  contrast  reduction  as  a  transmittance  uniformity  of 
76%. 


5.  Effect  Due  to  Differences  Between  Object 
and  Reference  Functions 

To  consider  the  contrast  reduction  of  the  correlation 
filter  caused  by  such  differences,  the  reference  function 
distributed  in  the  Fourier  plane  is  given  by 

gix.yi=  1  G„.«e  -  -  .  f2oi 

where  2 L,  x  2L,  are  the  dimensions  of  the  input 
functions.  G_  .  is  the  Fourier  coefficient,  and  1  is  the 
pixel  size  of  the  input  function.  However,  for  simplicity, 
a  pair  of  spatial  frequencies  i  u0.  rc.l  is  considered,  which 
is  given  bv 


To  obtain  a  simple  and  dear  physical  picture,  a  1  -D 
case -where  u=0  and  — 0  is  considered.  Thus  (27) 


reauces  to 


2sinc(2Lyr)siac^2E?M 

>•= - 1  '  (2S) 

S2  sinc'l  2L.r)  -h  -p  sine2 !  2L. ] 

The  average  contrast  y  can  of  course  be  obtained  by- 
using  (141  Figure  5  shows  the  average  contrast  7  as  a 
function  of  scale  factor  S  over  the  freouenev  band 


gft-X-  VI  “  fiiUg.  1q)C 


tCCi  1  r— “  1  sCCt  i 


wr*crc  u0  =  m0  —  and  ro  =  n0  r 

L. 


$c**$ 

Ft*.  5.  Average  conirasi  zs  a  function  of  texk 
frequence  bandwidth  i4  periods 


I5S 
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(0.twL  We  see  that  the  average  contrast  of  the  filter 
decreases  as  the  scale  factor  increases. 

On  the  other  hand,  if  the  input  function  is  rotated 
as  described  by  the  following  equation: 


mx.  vi = 


f xcosw-M  v  -r  ot  stay  t 

- ± - J 


(  -  x  sin  Q  4*  I  r + b)  cos  0\ 
x  rcct |  - - - - - j 

X  H  fit*  r 


(29i 


6.  Contrast  Redaction  oa  MF 

It  is  evident  that  a  matched  filter  can  be  constructed  by 
replacing  the  input  .unction  with  a  pinhole.  un. 
hx.  y — hi = AMx.  y — b)  as  shown  in  Fig.  1. 

The  contrast  of  MF  caa  be  written  as 

ZA  01 

*;=  - — — - —  1%/i 

•  4-- Cr‘ 

where  A  is  the  constant  amplitude  of  the  tilted 
reference  plane  wave.  Thus  for  j  MOD.  the  contrast  of 
the  MF  can  be  written  as 


ZAALJ-^lGAusj.  rsi  sine  [21.  Ju  -  api]  sincjlL^x — 
^i-i-t4LijL>Gju#.r!))sincf2L.<a-uwnsmc[2LJF-(ftQ|J ' 


tool 


where /  is  the  angle  of  rotation  of  the  input  function. 
The  o  .responding  Fourier  spectral  distribution  is 

H iu.v)= 4L.LJi’ciufj,  r0)  sinclL  Aq  sin  0 

-rpcosfftsiu2LIiecos0-psinffi£>2lc*r.  (30) 

where  p— r-u^sinl?— roeos0  and  q-u-u^cosff 
f  r6aaff.  Then  the  average  contrast  of  the  fitter,  for  a 
1-D  representation  is  given  by 

—  f'FcosfhsmcrZL.rsinyi 

sine*  ILyV  ~  sine*  -  IL^c  cos  01  aoc2  (2L,  sir,  Fj ' 


where  it  ts  assumed  that  ro=0. 

Three  average  contrasts  for  different  frequency 
bands  (4. 8.  and  16  periods)  are  plotted  in  Fig.  6  as  a 
function  of  rotational  ingle  6.  We  sec  that  the  larger 
the  frequency  band,  the  faster  the  average  -contrast 
decreases. 


Fig.  6.  A? 


air an  ;-rer  deftreai  frer  ctr.rr  ■•Mutsodiht  ji 
— .  pbwh  as  a  fcocuoo  of  rouuoo  anpr « 


Again  for  a  f-D  representation.  we  have 

2R  Rscpl^ft — r.lj ; 

R2  -sine1  [2Ljr  ~  r0}] ' 


where  fi  =  — .  From  this  equation  we  set  that 

4L.L.U0 

the  contrast  is  a  function  of  the  spatial  frequency 
variable  r.  where  ic.  is  the  earner  spatial  frequency  of 
the  reference  mane  wave.  Thus  the  contrast,  over  a 
given  frequency  land  ( -  -r  r^L  can  la  expressed  as 


5r  IRirircflL^r  -  r,)ji  . 
4.  R*  -r  sme*  [  _L*f  r  —  r0)j 


m 


To  obtain  an  optimum  /  ever  a  given  frequency 
bandwidth  (O.rJ.  wc  shall  search  for  the  best  It 
Figure  7  show’s  the  7  as  a  function  of  frequency  periods 
over  the  mage  (O.rJ.  Fipure  S  is  the  ptot  of  7  (over  8 
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159 


Parameter  A 

Fig.  8.  Average  contrast  of  a  matched  filter  (over  8  periods)  as  a 
function  of  the  intensity  ratio  R  =  A/(4LxLyG0) 


Table  1.  Optimum  contrast  of  a  matched  filter 


No.  of  periods 

2  4 

8 

12 

16 

20 

Optimum  y 

0.69  0.7 

0.71 

0.71 

071 

0.71 

Reference  'o-objcct 

69  12 

3 

1.9 

075 

0.5 

ratio 

It  can  Ua-  shown  that  the  following  well-known  in- 

equalL,  holds: 

\R(0,0rm(x,y)\2,  (37) 

where  |R{0, 0)|2  is  defined  as  the  correlation  peak 
intensity. 

As  before,  two  MODs  are  used  as  reference  and 
input  devices  in  the  correlator  to  analyze  the  degra¬ 
dation  of  the  output  peak  intensity.  As*1  "“mg  that  the 
input  function  /(x,  y)  is  a  scale  variant  .  .ie  reference 
function  g(x,  y),  we  have 

/.M-  (38) 

and  the  corresponding  Fou.ier  transform  is 
FM  ^  ^ 

(39) 

where  S  is  the  scale-change  factor.  Thus  the  correlation 
function  of  f(x,y)  and  g(x,y)  can  be  written  as 


periods)  as  a  function  of  A,  which  is  proportional  to  R. 
From  this  figure,  we  see  that  an  optimum  y=0.71 
occurred  at  R = 0.05.  As  an  example,  when  4 LxLy  -  7.2 

x 4.8 mm2,  then  —  si. 7,  That  means  the  intensity 
G 

reference-to-object  beam  ratio  is  about  3.  We  note  that 
over  a  different  number  of  periods,  the  optimum  y 
changes  very  slightly  with  R. 

The  corresponding  values  of  optimum  y  reference- 
to-object  beam  ratio,  and  the  number  of  periods  is 
tabulated  in  T  Me  1  for  4bxLy = 7.2  x  4.8  mm2.  From 
this  table  we  see  thai  the  average  contrast  of  the 
matched  filter  is  around  0.7  for  different  numbers  of 
periods  covered  by  the  MF.  Comparing  with  the  noisy 
JTC  filter  of  Figs  3  and  4,  we  see  that  the  contrast  for 
the  JTC  filters  are  higher  than  the  noiseless  MF,  if  tne 
surface  quality  of  the  device  is  better  than  /,  and  the 
uniformity  of  the  device  is  higher  than  j0%. 


Ks(x,y)=l  j  sinc(/w)  sinc(iu) 

•  (l  \  ■  (l 
x  sine  I  -u  jsmcl  -v 


^  ^  Q-\2nd(mu  +  rtv ) 


d 


x  z  Am  ne'2ns(mu+'"') e'2*lux+vy)du  dv .  (40) 


Using  the  convolution  theorem,  (40)  can  be  expressed 
as 

R,(x,y)  =  rs(x)rs(y)*  £  £  Am_nAm.,n. 


x  d 
where 


.,  m 

x  +  d\  y  -m 


y  +  d\--n 


(41) 


7,  Correlation  Degradation 

It  is  well  kr.cwn,  both  for  the  Vander  Lugt  correlator 
and  the  J1C,  that  the  resulting  output  correlation 
appears  as  a  bright  spot.  To  determine  the  perfor¬ 
mance  of  an  optical  correlation,  it  s  necessary  to 
measure  the  correlation  peak  intensity. 

Let  the  correlation  functions  of  f(x,y)  and  g(x,  y) 
be  given  by 

R(x, y)  =  j  J  F(u,v)G*(u,v)c'?m^vv)dudv. 


r,(x)  =  rect  -  *  red 


/ 

S’ 

I 

l 

S' 


x 

Vs 


1+S 


N,  ^(l-i 


<  x  < 


lxs 


0, 


(36) 


(42) 
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For  example,  a  letter  “A”  is  displayed  on  a  MOD 
device  as  the  reference  function  represented  by 
I  Am,„.  The  correlation  peak  intensity  R2(0,0)  of  “A” 

m.  a 

and  the  scale-changed  “A”  are  calculated  as  a  function 
of  the  scale  change  factor  S.  Again  using  a  1-D 
presentation  and  assuming  that  Am.„  =  1  for  ali  m  and  n, 
(40)  can  be  reduced  to 


R,  W= 


ao  J2 

J  — sinc(/n)sinc 

-  co  O 


x  'Z&~i2a"m,l^m“Qi2KUXdv, 

m  m' 


=  ri(x)*  £  8 


(43) 


Using  this  result,  the  correlation  peak  intensity  can  be 
computed.  Figure  9  shows  the  correlation  peak  inten- 
sity,  i?f(0,0),  as  a  function  of  the  scale-changed  “A”  as 
the  dotted  curve,  while  the  solid  curve  represents  i?2( 0) 
for  an  1  -D  representation. 

If  the  input  function  f(x,  y)  is  rotated  as  given  by 


Um 0=  I 

m,n  \  I 

' -xsind  +  ycos6-nd 


xrect 


(44) 


where  (9  denotes  the  angle  of  rotation  with  respect  to 
the  reference  function.  The  corresponding  spectral 
distribution  can  be  written  as 


F,(u,v)=  £  l2Am.  „  sine  [/(u  cos  0+p  sin  0)] 

tn.fl 

x  sine  [/( — u  sin  0  +  v  cos  0)] 

X  e-i2jtiml(Hcos9  +  i’sm0)e-i2>!>td(~ujin8  +  Dcos8) 


(45) 


48*48  Array—  ,  "A*  on  1 1x15  Array- 

Fig.  9.  Correlation  peak  intensity  as  a  function  of  the  scale 
change  factor  S 


Rotation  Angle  9=0.5  ° _ 5.5° 


Fig.  10.  Convolution  of  a  square  with  4=0.l27mm  and  its 
rotated  version 


F or  simplicity,  consider  me  correlation  of  the  reference 
function  and  its  rotated  version  only  on  the  y  axis  (i.e., 
u  =  0);  then  the  correlation  function  can  be  expressed 
as 


KrOO^cOO*  £  £  AmnAm._n. 

m.n  m’.n' 

x<5[y+d(msin0+/;cos0~n')],  (46) 


where  rc{y)  represents  the  convolution  between  a 
square  (with  side  1)  with  respect  to  its  rotated  version 
along  the  y  axis,  i.e., 


rt(y)=rect 


rM?) 


*rect 


xcos0+ysin0'\ 


l 


x  red 


-xsin0  +  ycos0\ 


l 


i 


(47) 


Figure  10  sketches  the  function  r,(y)  for  different 
rotation  angles,  0.5*,  5.5°,  and  10.5°.>rom  this  figure 
we  see  that  for  small  0,  rc(y)  can  be  approximated  by 


rc(y)  —  a 


0,  |  y\2l. 


(48) 


Again,  we  use  the  letter  “A”  for  computing  the 
correlation  intensity  R?( 0).  Figure  1 1  shows  the  corre¬ 
lation  peak  intensity  as  a  function  of  rotation  angle  0 
(dotted  curve). 

If  it  is  assumed  that  Am 1,  then  (46)  reduces  to 


—  rcC>’) %  £  <5[y  +  d(wsin0-r /1COS0  — «')],  (49) 

m.n.n’ 

which  is  essciuially  the  correlation  of  the  square  array 
and  its  rotated  function.  The  correlation  peak  intensity 
Rr( 0)  is  plotted  as  the  solid  curves  in  Fig.  11. 


Correlation  Peak  Intensity 
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Rotation  Angle  9 

48x48  Array —  'A*  on  1 1x15  Array— 


Fig.  11.  Correlation  peak  intensity  vs.  rotation  angle  0 


0.0  0.1  0.2  0.3  0.4  0.3 


Average  Value  AN 

Fig.  12.  Correlation  peak  intensity  as  a  function  of  average  value 
AN  for  Gaussian  ( — ),  uniform  ( — ),  and  exponential  (---) 
amplitude  noise 


8.  Noise  and  Correlation  Degradation 

By  referring  to  (10),  the  output  correlation  of  the  noisy 
input  object  with  the  reference  function  is  given  by 

R^,y)=  l  £  Am,nAm.,n.\NmJ^p- 

m.n  m'.n' 

y-d(n-n') 

j  J’ 

(50) 

where  A  is  defined  in  (48). 

If  only  amplitude  noise  is  considered  with  Am  „ 
=  Am>'„<  =  \,  the  above  equation  is  reduced  to 

m.n  m'.n'  L  * 

x  A  'y=* <P>],  (51) 

where  | Nmn\  is  given  by  (11). 

Similarly,  for  phase  noise  only,  the  output  correla¬ 
tion  function  becomes 


Rv(x,y)=  I  I  ei2'tPm-n  A 

m.n  m'.n 


l  . 


(52) 


where  el2,:Pm'"  represents  the  phase  noise.  Figure  12 
shows  a  set  of  correlation  peak  intensities,  f?2(0,0),  as  a 
function  of  the  average  value  AN,  for  Gaussian, 
uniform,  and  exponential  amplitude  noise  statistics, 
respectively  From  this  figure,  we  sec  that  the  variation 
of  peak  intensities  for  those  noise  statistics,  drop  rather 
rapidly  at  about  the  same  rate.  The  peaks  drop  to 
about  77%  when  AN =0.12.  The  average  value  of  0.12 


Random  Valus  Range  aP 

Fig.  13.  Correlation  peak  intensity  as  a  function  of  the  random 
value  range  4  P  for  Gaussian  ( — ),  uniform  ( — ),  and  exponen¬ 
tial  (---)  phase  noise  x 

means  the  maximum  departure  of  the  amplitude 
transmittance  of  a  MOD  pixel.  This  corresponds  to 
about  24%  of  the  unity  amplitude  transmittance, 
which  is  equal  to  0.19  optical  density. 

Similarly,  K2(0, 0)  as  a  function  of  the  random  value 
range  AP,  for  various  phase  noise  statistics  is  plotted  in 
Fig.  13.  It  is  seen  that  the  Gaussian  phase  noise  is  a 
reasonable  description.  When  the  value  range  is 
AP  =  1/2,  i.e.,  the  device  has  a  surface  quality  of  2/2,  the 
correlation  peak  intensity  drops  23%  from  the  unity 
correlation  peak.  Comparing  Figs.  12  and  13,  wc  see 
that  the  phase  noise  due  to  a  surface  quality  of  2/2  has 
the  same  effect  on  the  correlation  peak  intensity  as  an 
amplitude  noise  with  the  average  value  of  0.12.  This 
corresponds  to  a  transmittance  uniformity  of  88%  of 
the  MOD. 
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9.  Noise  Performance 


Assume  that  R  and  Rs  are  the  output  correlation 
functions  with  and  without  noise  respectively.  The 
output  signal-to-noise  ratio  can  be  defined  as 


5  E{R2} 

N~  E{(R-Rn)2}’ 


(53) 


where  E{  }  represents  the  ensemble  average.  However, 
in  an  optical  sense,  the  signai-to-noise  ratio  can  be 
written  as 


c  1  J  \R\2dxdv 

_ XY _ 

•  Af”n  \R~Rs\2dxdy’ 


(54) 
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where  (x,y)  represent  the  output  coordinate  system.  Fig.  14.  Signal-to-noise  ratio  S/A'  as  a  function  of  the  average 

Again  for  a  1-D  representation,  R  and  RN,  as  value  AN  for  Gaussian  amplitude  noise 

referred  to  in  (53),  are  given  by 


Similarly  for  phase  noise,  Gaussian  p1*58  Noise 


I R 


^y-d(n-n') 


-  T  cos27tP„  A 


+  (  T  sin27rP„  A 

\n.n 


> 


l 

■d(n  —  n) 


l 


(59) 


By  assuming  that  N„  and  P„  are  Gaussianiv  disturbed, 
the  output  signal-to-noise  ratios  for  amplitude  and 
phase  noise  disturbances  are  plotted  in  Figs.  1 4  and  1 5. 
respectively.  For  example,  if  dP^l/4  (the  surface 
quality  of  the  device  is  about  or  better),  the  signal- 
to-noise  ratio  should  be  greater  than  23  dB.  For  the 
same  SNR  in  Fig.  14,  0.07,  which  is  equivalent  to 

a  maximum  departure  of  14%  in  the  amplitude 
transmittance  of  the  device,  or  a  transmittance  uni¬ 
formity  of  93%. 


Fig.  15.  Signal-to-noise  ratio  S/iV  as  a  function  of  the  random 
vaiue  A  P  due  to  Gaussian  phase  noise 


10.  Comparison  of  VLC  and  JTC 

Let  us  now  look  at  the  modulation  indices  m  the  joint 
transform  filter  and  the  matched  filter.  If  the  input 
function  is  identical  to  the  reference  function  under  the 
same  intensity  illumination,  the  joint  transform  inter¬ 
ference  pattern  is  fully  modulated.  In  other  words,  the 
contrast  of  the  fringe  pattern  is  unity  throughout  the 
joint  transform  filter.  However,  to  estimate  the  modu¬ 
lation  of  the  Vander  Lugt  type  matched  filter,  the 
ensemble  average  contrast  is  utilized.  The  average 
contrast  of  the  fringes  was  found  to  be  about  0.7  for  an 
optimum  matched  filter  (i.c..  object-to-reference  beam 
ratio  of  unity)  and  since  the  average  modulation  index 
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of  the  joint  transform  hologram  is  higher  than  the 
matched  filter,  the  joint  transform  correlator  should 
provide  a  higher  autocorrelation  peak  intensity. 

For  the  Vander  Lugt  type,  it  is  required  that  the 
matched  filter  be  placed  back  in  its  original  recording 
position.  Practical  alignment  must  be  to  within  the 
order  of  about  0.01  mm  in  the  Fourier  plane.  However, 
for  the  joint  transform  correlation,  there  is  no  align¬ 
ment  requirement  for  the  position  of  the  filter  in 
Fourier  plane. 

The  space  x  bandwidth  product  is  an  important 
specification  for  an  optical  processing  system, 
although  it  never  reaches  the  specific  limit  in  practice. 
At  first  glance,  the  joint  transform  correlator  requires  a 
space  x  bandwidth  product  twice  that  of  the  Vander 
Lugt  correlator,  where  in  Fig.  1  a  pinhole  for  the 
construction  of  the  matched  filter  can  be  seen.  There¬ 
fore.  the  space  x  bandwidth  product  of  the  Vander 
Lugt  is  indeed  lower  than  that  of  the  joint  transform 
correlator.  The  key  issue  is.  however,  not  the  space 
x  bandwidth  product,  but  that  one  can  fully  utilize  the 
product  provided  by  the  correlator. 

There  is  aiso  a  difference  between  the  Vander  Lugt 
and  joint  transform  correlators  in  multiplexing,  or 
multiple  object  recognition.  In  a  Vander  Lugt  correla¬ 
tor,  several  functions  are  placed  in  the  input  plane  and 
a  set  of  metched  filters  placed  in  the  Fourier  plane  for 
the  multiplexing  operation.  However,  in  a  joint  trans¬ 
form  correlator,  instead  of  using  multiple  filters  m  the 
Fourier  plane,  multiple  reference  functions  can  be  put 
in  the  input  plane  of  the  correlator  for  multiple  object 
recognition-  In  addition,  a  multichannel  joint  trans¬ 
form  correlator  can  be  constructed  by  using  a  special 
multiple  Fourier  transform  lens  or  an  array  of  small 
lenses 

As  mentioned  previously,  the  noise  in  most  optical 
correlation  systems  is  basically  of  multiplicative  type, 
and  is  introduced  either  by  the  input  object  or  by 
spatial  light  modulators  used  in  the  optical  system.  We 
have  shown  that  the  noise  effects  on  the  output 
correlation  peak  for  both  Vander  Lugt  and  joint 
transform  correlators  are  about  the  same. 

Wc  should  emphasize  that,  for  real-time  operation, 
the  joint  transform  correlator  is  simpler  and  more 
straightforward  to  operate  than  the  Vander  Lugt 
correlator.  A  rcai-time  joint  transform  correlator  is  a 
one-step  operation,  if  the  appropriate  spatial  light 
modulators  are  introduced  in  the  input  and  the 
Fourier  planes.  The  advantage  of  the  one-step  oper¬ 
ation  is  that  prefabrication  of  the  correlation  filter  is 
not  required.  However,  for  the  Vander  Lugt  correlator, 
a  two-step  process  is  required  m  which  a  matched  filter 
must  be  prefabricated.  Notice  that  a  computer- 
generated  matched  filter  can  be  directly  implemented 
in  a  spatial  light  modulator,  such  as  a  MOD.  Although 


& 


Table  2.  Comparison  U  Vander  Lugt  and  joint  transform 
correlators 


Joint  transform 

Vander  Lugt 

Modulation  index 

7=1 

7=0.7 

Alignment 

No 

crucial.  Jr=0.01  mm 

Space  x  bandwidth 

Greater 

smaller 

Multiplexing 

Set  of  reference 
functions 

Set  of  matched  filters 

Rea!  time  operation 

One-step 

Two-step 

Noise  performance 

The  same 

The  same 

this  makes  a  one-step  Vander  Lugt  correlator  possible, 
complicated  computations  must  be  performed  to  gene¬ 
rate  the  filter. 

We  summarize  the  comparison  of  the  Vander  Lugt 
and  the  joint  transform  correlator  in  Table  2.  Thus,  w  e 
see  that  the  joint  transform  correlator  would,  tn 
general,  perform  better,  particularly  for  real-time 
implementation. 


11.  Conclusion 

A  generalized  optical  correlator  has  been  used  for  the 
Vander  Lugt  and  the  joint  transform  correlators. 
Under  such  a  unified  description,  the  analyses  of  to  the 
noise  disturbances  and  the  differences  between  the 
input  and  the  reference  functions  are  valid  for  both 
architectures.  To  consider  the  effects  due  to  noise, 
various  amplitude  and  phase  models  were  used. 
Magneto-optic  spatial  light  modulators  wrere  used  for 
the  system  performance  calculation. 

The  contrast  of  the  correlation  filters  for  both 
correlators  were  analyzed.  We  have  shown  that  the 
contrast  of  the  correlation  filter  was  reduced  by  the 
presence  of  noise.  The  effect  of  scale  change  and 
rotation  of  the  input  function  on  the  contrast  of  the 
filter  was  also  calculated.  The  correlation  peak  inten¬ 
sity  under  the  influence  of  noise  and  differences  between 
the  input  and  reiercncc  function  was  aiso  studied.  We 
have  found  that  the  decrease  of  the  correlation  peak 
depends  on  the  type  of  noise  and  the  vaiue  range  of  the 
noise.  The  scale  change  and  rotation  of  the  input  aiso 
degrade  the  correlation  peak  intensity.  By  comparing 
the  performances  of  the  Vander  Lugt  and  joint  trans¬ 
form  correlators,  wc  have  shown  that  the  joint  trans¬ 
form  correlator  is  easier  to  construct  and  offers 
advantages  in  real-time  implementations. 
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GRAY  LEVEL  PSEUDOCOLOR  ENCODING  USING  A 

LIQUID  CRYSTAL  TELEVISION 
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SUMMARY" :  A  real-time  gray  level  intensity  pseudocolor  encod¬ 
ing  method  utilizing  a  liquid  crystal  television  (LCTV)  is  pre¬ 
sented.  The  LCTV  is  used  under  two  different  polarizer/analyser 
combinations  to  generate  3  positive  image  and  a  negative  image  of 
an  object.  The  images  are  also  encoded  with  two  different 
primary  colors  by  positioning  the  color  filters  at  the  two  optical 
paths  respectively.  The  superposition  of  these  images  produces  a 
gray  level  pseudocolor  encoded  image.  Discussions  on  the  proper¬ 
ties  of  the  LCTV  is  provided.  Preliminary  experimental  results 
are  also  demonstrated. 


Codage  en  fausses  couleurs  des  niveaux 
de  gris  par  un  systeme  de  television 
a  cristal  liquide 

RESUME  :  On  presume  unc  methode  de  codage  en  fausses 
cot-.-urs  de  niveaux  de  gris  utilisant  cn  temps  reel  un  systems  de 
television  a  enstai  liquide.  Le  systeme  a  cnstal  liquide  cst  utilise 
avee  deux  combinaisons  analyscur-polariscur  pour  donner  dc 
1‘objct  uric  image  positive  ct  unc  image  negative.  Les  images  sont 
codecs  awe  deux  couicurs  dc  base  en  plagant  dcs  fiitrcs  sur  les 
deux  trajers  optiques.  La  superposition  de  ccs  images  donne  un 
codaec  des  gris  en  fausses  couicurs.  Les  propriclcs  du  systeme  a 
crista!  'iausde  sont  discutccs  ct  on  donne  quclqucs  rcsultats 
experimemaux  prcliminaires. 


i.  —  ISTRODLCTIOS 

Most  of  the  optical  images  obtained  in  various 
scientific  applications  are  gray  level  intensity  images, 
for  example,  the  scanning  electron  microscopic  un- 
ages,  muitispectral  band  aerial  photographic  images. 
X-ray  transparencies,  endoscopic  images,  etc.  How¬ 
ever.  human  perceives  variations  in  color  better  than 
those  in  gray  ieveis.  In  other  words,  a  color-coded 
image  provides  better  visual  discrimination  (1 ). 

Density  pseudocolor  encoding  by  half  tone  -.greens 
implementation  with  a  coherent  optical  processor 
was  first  reported  by  Liu  and  Goodman  (2J.  and 
later  with  a  wmte-light  processo-  by  Tai  a  ul  ;3J. 
Based  on  similar  principles,  many  optical  architec¬ 
tures  have  been  reported  recently  [4-8]. 

The  recent  appearance  of  a  low-cost 
1  =  S  iOO )hlack-and-\vhite  liquid  crystal  television 
(LCTV)  has  attracted  a  great  deal  of  attention  to 
explore  its  usetulness  for  real-time  optical  signal 
processing  [°J  We  demonstrated  the  application  ot 


LCTV  to  white-light  processing  [  SOJ.  which  included 
the  optical  pseudocolor  encoding,  i.e.  different  grat¬ 
ing  structures  resulted  in  different  colors.  A  direct 
method  to  pseudocolor-encode  light  intensity  using 
the  LCTV  was  also  reported  recently  (111-  In  this 
method,  a  light  intensity  distribution  is  recorded  by  a 
video  camera,  in  which  light  intensity  is  transduced 
to  electric  signal.  The  electronic  video  signal  from 
the  camera  is  then  fed  to  the  LCTV  to  produce  the 
voltage  difference  across  the  iiqutd  crystal.  A  color 
pattern  corresponding  to  the  applied  voltage  will 
appear  due  to  the  birefringence  of  the  molecules  of 
liquid  crystal.  This  muiu»il:ij.  however,  has  a 
disadvantage  that  the  ^oior  composition  is  fully 
determined  by  the  iiquid  crystal  display  structure 
and  the  orientation  of  polarizers. 

in  this  paper,  we  demonstrate  a  new  method 
similar  to  the  halftone  screen  technique  [2|.  The 
LCTV  generates  two  distinctive  primary  color  im¬ 
ages  with  contrast  reversed.  The  superposition  ol 
the  two  .mages  produce  a  pscudocoior  encoded 
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ation  of  the  incident  linear  polarized  I'ght  is  rotated 
through  90'  by  the  LC  material,  hence  no  light 
passes  through  the  analyser.  When  the  voltage 
across  the  ceil  is  fully  applied,  the  LC  molecules  will 
align  in  the  direction  of  the  applied  electric  field, 
hence  the  polarization  direction  of  light  passed 
through  is  not  rotated  and  there  is  totai  transmission 
of  light  through  the  analyser.  If  this  parallel  analyser 
generates  a  positive  image,  the  orthogonal  analyser 
wouid  produce  a  negative  image.  Chao  and  Liu  [12J 
reported  a  pseudocolor  technique  that  superimposed 
red.  green  and  blue  color-encoded  images  obtained 
with  parallel.  45*  and  90*  oriented  analysers  respect¬ 
ively. 

The  difference  between  the  proposed  technique 
here  and  that  of  Chao  and  Liu  [12|  is  that  we  utilize 
the  hybrid  field  effect  of  the  liquid  crystal  ma¬ 
terial  |  1 3)  <  twisted  nematic  field  effect  and  birefrin¬ 
gence)  while  they  basically  utilized  the  twisted 
nematic  field  effect  only.  Such  a  difference  can  be 
observed  easily  from  the  experimental  setup,  in 
which  we  used  a  common  analyser  and  variable 
polarizers,  while  Chao  and  Liu  used  variable  analys¬ 
ers  and  a  common  polarizer. 

The  voltage  on  each  pixel  is  determined  by  two 
factors  :  a  brightness  control  switch  controls  the  bias 
voltage  across  all  the  pixels  on  the  screen  and  the 
video  nput  sign-ii  controls  the  voltage  across  individ¬ 
ual  pixels.  Neglecting  the  detailed  analysis  of  the 
physical  phenomenon  of  birefringence,  we  may 
develop  an  empirical  model  as  follows.  When  input 
signal  is  applied  to  a  pixel,  the  LC  molecules  snows 
the  birefringence  [I  I  j.  The  polarization  axis  of  inci¬ 
dent  light  is  rotated  by  an  angle  of  90’  -  fl5  - 
where  e)s  is  a  function  of  the  bias  voltage  and 
is  a  function  of  the  input  signal. 

The  two  operating  modes  of  the  LCTV  we  have 
used  in  the  experiment  are  as  shown  sn  figure  4.  In 
the  first  mode  as  shown  in  figure  4uo.  the  irradiance 
transmitted  is  given  by 

/,  =  cos"  (90*  -  «}  -  (),,) .  (!) 

In  the  second  mode  as  shown  :n  figure  4th)  the 
polarizer  .  T  is  rotated  by  an  angie  of  -  Q. wnai. 
where  is  the  maximum  angle  o(  that  the 

LCTV  can  achieve.  In  this  experiment  (Radio  Shack 
Vision  3  LCTV  was  used)  we  found  0  ,,_4l  =  !3*. 
The  transmitted  irradiance  of  the  second  mode  ;s 
given  by 

/_-  =  cos*  ( 90‘  -  ) .  (2) 

Suppose  0,  c  i  /» =  13*  and  O', 

(hen 

(3) 
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PI  LCTV  PZ 


(a) 


P2  LCTV  P3 


lb) 


Fie.  -4  —  The  wo  operating  modes  of  the  LCTV  used  in  the 
experiment.  PI.  PZ.  P3  are  polarizers.  The  arrows  show  : he 
directions  of  the  poianzers. 


where 

/ f  =  cos"  (90*  -  flj)  =  dark  .  (4) 

/,  =  cos"  (90*  -  6b  -  13* )  =  bright .  (5) 

When  =  13*. 

/:>/;.  (6) 

where 

/:  =  cos"  (90*  —  0b  -  13* )  =  bright  .  (7) 

/.  =  cos"  (90'  -  0b)  =  dark  .  (8) 


WHITE 

LIGHT 

SOURCE 


rn  u  p;  es  lctv  =»:  oitfosiR 

("ig.  5  —  optic  ./  archttti'ure  nf  gras  level  pseudocolor  encoding. 
/.I.  1.2  are  enihmanng  lensi t ,  (  I  .  C  t  .  are  color  filters  .  I’l .  P2. 
are  poianzers .  US  it  a  hi  ws  tphuer 


!:  can  be  seen  that  if  'h  is  suffictcnilv  small 


(  iK  --■?  ~  "  )  ■  then  the  first  and 


Cvonu  niouc 


gc.^rate  high  contras!  positive  anu  neuame  image 
respectively  in  other  words,  the  transmitted  ir- 
radiances  giver.  in  the  two  modes  are  tn  reversed 
contrast. 


4.  —  KXPKRIMKNTS 

Indeed.  a;i  spatial  light  modulators  based  o.i  light 
poiari/ation  modulation,  such,  a.s  ntagr.eto-opnc  spa- 
MSI  lllZii;  iator  .  !>uuiu  ervstai  light  %  il\r  jlph 

•mcro-channe!  spatial  Imht  modulator.  ;n  addition  (■- 


the  LCTV.  can  be  used  m  this  method.  We  select  to 
use  the  LCTV  m  this  experiment  ir.ainls  due  to  its 
features,  namely,  electrically  addressing  and  low- 
cost  . 

The  optical  setup  is  as  shown  in  nqureF.  The 
combinations  of  polarizers  iP\.  /'3  )  and  IP 2.  /'3  >. 
as  discussed  -.n  the  previous  section,  simultaneously 


a  positive 


anu  a  negative  image. 
:d  on  the  diffuser.  Two 


winch  are  then  superimposed  on  the  uutuscr.  I  wo 
distinctive  primary  color  filters  Cr  and  CF-  arc 
positioned  at  the  two  optical  paths  respectively.  The 
pseudoeolor-eiicoued  image  can  then  nc  observed 
on  the  diffuser  hetvnd  the  polarizer  P3 

•\  prehmmarv  experiment  has  been  periormed  to 
vents  the  promised  method.  In  the  experiment. 


.  *•»  » 
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CF,  was  a  red  filter  and  CFZ  was  a  green  filter. 
P 1  and  P 2  were  orientated  at  0*  and  -  13*  with 
respect  to  P3  accordingly.  We  first  used  a  three-level 
gray  scale  chart  as  a  test  object.  When  the  green 
light  was  blocked,  a  positive  red  image  was  observed. 
Similarly,  a  negative  green  image  was  observed  by 
blocking  the  red  light.  The  gray  scale  chan  and  the 
color  encoded  output  are  shown  in  figure  6.  Three 
distinctive  colors  were  observed  corresponding  to 
the  three  gray  levels.  Another  result  of  a  X-ray 
transparency  is  demonstrated  in  figure  7.  However, 
the  intermediate  color  (yellow)  was  dim  compared 
with  the  red  and  green  color.  This  might  be  caused 
by  the  low  contrast  of  the  original  pattern  generated 
on  the  LCTV, 


5.  _  CONCLUDING  REMARKS 

A  gray  level  pseudocolor  encoding  technique  that 
utilizes  modulation  of  polarization  in  a  spatial  light 
modulator  has  been  discussed.  Though  the  proposed 
technique  is  applicable  to  many  SLMs  such  a 
MOSLM.  LCLY.  MCSLM.  etc.,  a  preliminary  ex¬ 
periment  has  been  demonstrated  using  a  LCTV.  The 
main  advantage  of  this  method  is  that  it  is  an 
economic  real  time  technique  with  simple  optical 
architecture.  However,  with  the  LCTV  we  have 
worked  with,  the  low  contrast  ratio  of  the  device 
limits  the  dynamic  range  of  color  content.  Also, 
there  is  a  loss  in  details  of  the  image  due  to  the 
quantization  of  gray  levei  and  the  limited  resolution 
of  the  LCTV.  With  further  improvement  in  the 
quality  of  LCTVs.  this  real  time  pseudocolor  encod¬ 
ing  technique  may  be  very  useful  in  medical  and 
industrial  applications. 
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White-light  joint-transform  correlator 
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oat  process  are  described,  arri  experimental  reswis  are  fives. 


The  joint-transform  correlator  (JTC)  was  proposed  by 
Weaver  and  Goodman1  for  optically  convolving  a  pair 
of  two-dimensional  functions  without  using  a  complex 
spatial  filter  in  the  frequency  plane.  Thus  the  align- 
ment  problem  of  the  spatial  filter  in  the  traditional 
Yander  Lugt  system2  is  avoided-  Recently,  by  incor¬ 
porating  some  spatial  light  modulators  in  the  optical 
system,  the  JTC  was  applied  to  real-time  pattern  rec¬ 
ognition.3  Because  there  are  many  advantages,  in¬ 
cluding  a  higher  signal-to- noise  ratio,  associated  with 
the  white-light  optical  processing  system,'  we  investi¬ 
gate  here  the  possibility  of  utilizing  white-light  illumi¬ 
nation  in  the  JTC- 

Consider  the  JTC  shown  in  Fig.  1,  in  which  Fig.  i  (a) 
is  the  optical  system  for  recording  the  joint  - transform 
hologram  (JTH)  and  Fig.  I{fe)  is  the  readout  optical 
system.  Usually  these  two  systems  are  connected  us¬ 
ing  a  beam  combiner.  Light  emitted  by  the  source 
point  in  plane  P,,  is  collimated  by  lens  L-;  to  shine  on 
the  objects  (both  reference  and  input  objects)  on  plane 
Pi.  A  square-law  detector  is  placed  in  frequency 
plane  Pj  to  record  the  joint  Fourier-transfonn  spec¬ 
trum  of  the  input  objects  ami  the  reference  object. 
Let  a  point  source  at  (x„  y,)  in  the  source  plane  slluini- 
nate  the  objects  with  amplitude  transmittance 

Kxj.Vi)  =  a,(xj  -  x„y,  -yr) 

+  cdxi  -  X,  -  X,  V;  -  y,  “  V),  (1) 


where  a,  is  the  reference  object  and  a*  is  the  input 
object  to  be  detected.  A  phase  grating  Gi  with  its  lire 
structure  parallel  to  the  x-  axis  is  placed  in  contact 
with  the  object.  A  first-order  joint-transform  spec¬ 
trum  (JTS)  is  recorded.  If  we  assume  linear  record¬ 
ing,  the  amplitude  transmittance  of  the  recorded  JTH 
can  be  written  as 
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where  A.  -  iAJexp(£,^,.)  and  A  -  ! Adeipti^i  are  the 
Fourier  transforms  of  a,  and  oe„  |-j  denote  j(x;  +  Fjx„ 
Fii/XFi,  iyt  +  FyJFi  —  AF^ii/Xfhj,  X  is  the  wave¬ 


length  of  the  light,  and  f%  is  the  spatial  frequency  of  the 
grating.  From  Eq.  (2)  we  note  that  the  spectrum  is 
dispersed  into  a  rainbow  color.  For  the  wavelength  X 
the  JTS  is  centered  at  x»  =  —F^xJF%  and  y-  »  —Fay */ 


F.+XFaA. 

In  order  to  record  the  JTS  successfully,  certain  co¬ 
herence  requirements  should  be  met.  The  spatial  co¬ 
herence  requirement  is  discussed  first.  For  simplic¬ 
ity,  if  we  assume  that  the  input  object  and  the  refer¬ 
ence  object  are  identical.  Eq.  (2)  becomes 


iAx^  ys)  =  2L44-JF.  J  + 


+  2r  (y,  +  F#JFX  -  Siy.)  jj-  (3) 

Let  a  line,  which  is  parallel  to  the  line  goiag  through 
the  center  of  both  objects,  in  the  source  pla-  a  be  called 
a  parallel  line.  Suppose  that  the  length  of  projection 
of  the  spatiaiiy  incoherent  source  on  the  parallel  'doe  is 
L,.  Summing  the  contributions  from  the  whole  illu¬ 
minating  source,  use  fringe  pattern  would  he  wiped 
out  if  the  pattern  produced  by  the  source  point  at  LJ2 
is  'laterally  shifted  by  one  fringe  period  with  respect  to 
that  produced  by  the  source  point  at  —LJ2.  This 
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Fig.  2.  JTS  for  estimat'  •  i  of  AX. 


corresponds  to  the  condition  (X2  +  Y1)112  =  \/{LJFx). 
Since  \/{LJF\)  is  the  coherence  length  in  the  direction 
of  the  parallel  line,  we  note  that,  in  order  to  produce 
the  fringe  pattern  with  considerable  contrast,  the  sep¬ 
aration  between  two  objr  •,  is  should  be  much  smaller 
than  the  lateral  coherence  length. 

For  simplicity  in  discussing  the  temporal  coherence 
requirement,  we  assume  that  a  white-light  point 
source,  at  the  origin  of  the  source  plane,  is  illuminating 
the  objects.  In  this  condition,  Eq.  (3)  can  be  simpli¬ 
fied  t  read 


tj(x^y2)  =  2Ur[-]|2 


Js 


1  +  C<Y'-S3p 

+  2t r  (y2  -  \Frjl) 


(4) 


Suppose  that  the  spatial  bandwidth  of  the  object  in 
the  *2  direction  is  A fx  and  that  in  the  y<i  direction  is 
A fy.  Referring  to  Fig.  2,  we  can  estimate  the  spectral 
bandwidth  AX  that  contributes  to  i"ie  final  fringe  pat¬ 
tern.  For  the  spatial  spectrum  of  the  object  due  to  X  + 
V2AX  to  be  separated  from  that  due  to  X  -  '/2AX,  we 
have 


AX  =  XA(y/„  (51 

where  A f>  «  f\  is  r  mined,  Since  the  fringe  pattern 
for  each  wavelengtn  a  13  centered  it  x2  -  0  and  >’2  = 
V1/F2,  the  dispersion  distance  cxhFtfx  should  be  small¬ 
er  than  tho  period  CKFz/Y)  of  the  fringe  pattern  in  the 
y2  direction  for  the  filial  fringe  pattern  not  to  be  wiped 
out  completely.  This  leads  to  the  criterion 

<  1,  (6) 

where  Eq.  (51  is  utilized.  For  example,  of  an  object  A/, 
=  10  mm~!,  the  ufi-axis  distance  Y shomd  be  smalle1- 
than  0.1  mm.  Thus,  to  nave  a  fringe  pattern  with  high 
contrast,  the  centers  of  both  the  input  and  tin  refer¬ 
ence  objects  should  lie  on  a  line  parallel  to  the  x\  axis 
In  this  condition,  Eq.  (4)  becomes 

1  +  cos/* 2ir  Xo')  >  (7) 

\  ^  -J] 

where  [•)  denotes  [xo/\Fi,  (y2  -  XP'/il/Xfb).  To  esti¬ 
mate  the  number  of  fringes  that  are  available  for  the 
readout  process,  assume  that  the  object  has  a  relative¬ 
ly  flat  spatial  spectrum.  Integrating  over  the  spec- 


M*2>  >'2)  =  2KMh 


trum  from  X  -  !/,AX  to  X  +  */2AX,  the  cosine  term  in  Eq. 
(7)  becomes 


A  + 


A  cos(2*J^)dX 


=  AX  sinc^^~  Xo^cos^r  x,^  *  (8) 

where  AX  «  X  or  A ,/y  «  fi  is  assumed.  From  Eq.  (8), 
we  find  ‘hat  the  number  of  fringes  under  both  the 
main  lobe  and  the  first  sidelobes  of  the  sine  func  .on  is 
4X/AX  or  4/i/A/y-  For  fi  =  80  mm-!  and  A/v  =  10 
mm"1,  the  number  of  fringes  would  be  32. 

In  the  readout  process  the  optical  system  shown  in 
Fig.  1(b)  is  used.  White  light  from  a  point  source  at 
the  origin  of  plane  P</  is  collimated  by  lens  L3.  Two 
identica’  gratings,  of  spatial  frequency  fu  are  situated 
on  planes  Pj  and  P4  with  their  line  structures  parallel 
to  the  x  axis.  A  rectangular  aperture  rect(x,3/ 
LJrectCys/Lv)  on  plane  P3  limits  the  spatial  extent  of 
the  plane  wave.  Owing  to  the  dispersion  effect  of 
grating  G2,  the  first-order  wave  of  each  wavelength  X 
would  appear  in  the  region  Tect(xJL%)rect{y4  —  XFYi)/ 
Ly)  on  plane  P4,  where  the  diffraction  effect  on  the  rim 
of  the  rectangle  is  neglected.  When  it  is  incident  upon 
grating  G3,  this  wave  is  redirected.  The  wave,  propa¬ 
gating  in  the  direction  parallel  to  the  optical  axi  ■* 
retained  to  shine  on  the  JTH.  Let  the  readout  w. . 
completely  overlap  the  JTS.  At  plane  P,  the  intensity 
'  -ribution  of  the  correlation  spots  would  be 


X  exp (  ±i2ir  — — -  x, jdX' 


XT, 


exp|  —  I27T  — — -  x, 
A Jr  2 


X  expj  — i2r,  — _  ■,  |dx,dy2 

At  T 


(9) 


where  1+  (/_)  denotes  the  intensity  distribution  of  the 
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Fig.  3.  JTH  illumir.  led  by  a  wave  with  a  wavelength  * 
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Fig.  4.  (a)  Reference  and 
input  objects,  (b)  the  JTS, 
(c)  correlation  peaks  at  the 
output  plane. 


desired  correlation  spot  in  the  region  where  x  >  0  (x  < 
0),  \i  is  the  longest  wavelength,  As  is  the  shortest 
wavelength,  and  Eq.  (7)  is  used.  If  a  source  of  finite 
size  is  utilized  instead,  the  output  intensity  distribu¬ 
tion  would  be  the  convolution  of  Eq.  (9)  with  the 
source  distribution.  Equation  (9)  is  complicated  and 
needs  further  investigation.  Here,  we  give  a  heuristic 
explanation.  Figure  3  shows  the  JTH  illuminated  by 
a  wave  with  wavelength  A.  We  take  Lx  =  A FyAfx  and 
Ly  =  XFi&fy,  where  A  is  the  average  wavelength.  Ow¬ 
ing  to  the  finite  size  of  the  readout  wave,  the  linear  size 
of  the  resolution  element  is  AFj/(\FoAfx)  in  the  x  di¬ 
rection.  Similarly,  it  is  XFy/^XFyAfy)  in  the  y  direc¬ 
tion.  Since  the  spatial  period  of  the  fringes  in  the  Xy 
direction  varies  from  (A  +  lly\Afylf\)FylX  for  the  top  of 
the  JTH  to  (A  -  l/y\A fylf\)Fy/X  for  the  bottom  of  the 
JTH  in  Fig.  3,  the  center  of  the  desired  correlation 
peak  would  be  at  x  =  XFy/Fy,  with  a  slight  broadening 
XFy\Afy/(Fy\f\)  owing  to  nonuniformity  of  the 
fringes.  Generally,  the  combined  broadening  effect 
due  to  both  nonuniformity  of  the  fringes  /.  the  JTH 
and  diffraction  is  much  smaller  than  the  distance  be¬ 
tween  the  correlation  peaks.  For  instance,  if  A fx  =  A fy 
=  10  mm"1  and  Fy  =  Fy,  the  linear  size  of  the  correla¬ 
tion  spot  due  to  the  combined  effect  is  approximately 
0.2  mm.  Hence,  after  summing  the  contributions 
from  all  the  spectral  elements,  the  size  of  the  white- 
light  correlation  spot  stays  comparable  with  that  due 
tc  the  wave  with  average  wavelength  A. 

In  the  experiment  a  mercury-arc  source  is  imaged  by 
a  condenser  lens  onto  a  pinhole  of  diameter  15  nm. 
Since  the  focal  length  of  the  collimating  lens  is  30  cm. 
the  lateral  coherence  length  would  be  11  mm.  The 
letters  in  Fig.  4(a)  are  photoreduced  on  Kodak  SO-253 
film  and  become  transparent.  On  the  left-hand  side, 
the  letter  K  serves  as  the  reference  object,  while  the 
four  letters  on  tne  right-hand  side  are  the  input  ob¬ 
jects.  The  letters  K.  K,  and  Z  are  aligned  to  lie  on  a 
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line  parallel  to  the  axis.  The  distance  between  the 
reference  object  and  the  letter  Z  is  4.5  mm,  which  is 
smaller  than  half  the  lateral  coherence  length.  In 
order  to  meet  the  requirement  A fy  «  /j,  the  spatial 
frequency  of  the  phase  grating  Gi  is  taken  to  be  80 
mm'1.  The  focal  length  of  lens  Ly  is  60  cm.  Figure 
4(b)  shows  the  JTS  in  which  the  fringes  under  both  the 
main  lobe  and  the  first  sidelobes  of  the  sine  function 
are  approximately  30  lines.  In  the  readout  process, 
the  diameter  of  the  white-light  source  is  0.2  mm,  and 
the  focal  length  of  the  collimating  lens  L:j  and  the 
Fourier-transform  lens  L4  is  30  cm.  Parallel  gratings 
G2  and  G3  are  identical  to  grating  Gi  and  are  separated 
by  distance  Fy.  Shown  in  Fig.  4(c)  is  the  intensity 
distribution  of  the  output  at  plane  P.  The  bright  dots 
on  each  side  of  the  figure  are  the  correlation  peaks  of 
the  reference  letter  K  and  the  input  object  K  [to  the 
left  of  letter  Z  in  Fig.  4(a)].  There  is  no  output  for  the 
letter  K  that  is  above  the  letter  Z  because  relation  (6)  is 
severely  violated.  To  identify  the  letter  K  that  is 
above  the  letter  Z,  we  have  to  rotate  the  input  plane  (or 
grating  GO  about  the  optical  axis  until  the  line  going 
through  the  center  of  the  reference  object  and  that  of 
the  input  object  K  is  parallel  to  the  line  structure  of 
grating  Gi.  This  suggests  the  possibility  of  real-time 
two-dimensional  identification  of  the  input  objects. 
When  spatial  light  modulators  are  placed  in  planes  P\ 
and  Po,  a  two-dimensional  correlation  pattern  can  be 
produced  with  bright  correlation  spots  at  their  corre¬ 
sponding  positions  if  the  input  plane  and  the  output 
detector  are  rotated  synchronously. 

In  summary,  we  have  demonstrated  both  theoreti¬ 
cally  and  experimentally  the  possibility  of  utilizing 
white-light  illumination  in  the  JTC.  When  the  sepa¬ 
ration  between  the  reference  object  and  the  input  ob¬ 
ject  is  much  smaller  than  the  lateral  coherence  length, 
and  the  line  joining  both  objects  is  parallel  to  the  line 
structure  of  the  grating,  the  JTH  will  1  ave  good  fringe 
contrast.  By  increasing  the  spati frequency  of  the 
grating,  the  broadening  effect  of  the  correlation  spots 
due  to  nonuni.  irmity  of  the  fringes  can  be  kept  small 
in  the  readout  process.  Since  each  readout  process 
only  gives  the  information  about  one  line  in  the  object 
plane,  in  order  to  obtain  a  two-dimensional  correlation 
map,  both  the  input  and  the  output  planes  should  be 
continuously  rotated. 

We  acknowledge  support  by  the  U.S.  />  rmy  Missile 
Command  through  he  U.S.  Army  Research  Office 
under  contract  DA  '  L03-87-  0147. 

Y.  S.  Cheng  >'■  t  leave  from  Institute  of  Optical 
Sciences,  Natioi  1  Central  University,  Chung-Li,  Tai¬ 
wan  32054,  ’bin;*. 

References 

1.  C.  8.  Weaver  and  J.  \V.  Goodman.  Appi.  Opt  5,  1248 
(1966). 

2.  A,  ,7ander  Lugt,  IEEE  Trans.  Inf.  Theory  IT-10,  139 
(1964). 

3.  F.  T.  S.  Yu  and  X  J.  Lu.  Opt.  Commun.  52, 10  (1984). 

4.  F.  T.  S.  Yu,  White-Light  'Jphcal  Signal  Processing  (Wi¬ 
ley,  New  York,  1985),  p.  25. 


APPENDIX  10.8 


High-Efficient  JTC 


a  reprint  from  Optics  Letters 


68 

1029 


High-efficiency  joint-transform  correlator 

Francis  T.  S.  Yu 

Department  of  Electrical  Engineering,  The  Pennsylvania  State  University,  University  Park.  Pennsylvania  16802 

Eddy  C.  Tam 

Department  of  Physics,  University  of  North  Carolina  at  Charlotte,  Charlotte.  North  Carolina  28223 

Don  A.  Gregory 

U.S.  Army  Missile  Command,  Research  Directorate.  Redstone  Arsenal,  Alabama  35898-5248 
Received  December  11, 1989;  accepted  June  25, 1990 

A  technique  that  increases  the  efficiency  of  a  conventional  joint-transform  correlator  is  proposed.  By  modifying 
the  joint-transform  power  spectrum  of  the  input  objects  by  using  a  spatial  sampling  method,  the  readout  light  as 
well  as  the  physical  area  of  the  square-law  detector  can  be  fully  utilized.  As  a  result,  the  output  correlation  intensity 
can  be  substantially  increased.  Cases  of  using  coherent  and  partially  coherent  readout  light  are  discussed,  and 
experimental  results  are  presented. 


In  a  joint-transform  correlator1-2  (JTC)  the  spectra  of 
two  input  u  ze  functions  fix,  y)  and  gix,  y)  are  first 
recorded  b.  -  square-law  detector  (e.g.,  a  liquid-crys¬ 
tal  television,  a  microchannel  spatial  light  modulator, 
a  charge-coupled-device  camera,  or  photographic 
film).  A  plane  wave  is  then  applied  to  read  out  the 
joint-transform  power  spectrum,  producing  the  fol¬ 
lowing  complex  light  field  at  the  output  plane: 

u(x,  y)  =  fix,  y)  ®  fix,  y)  +  gix,  y)  ®  gix,  y) 

+  fix,  y)  ®  gix  +  2x0,  y)  +  gix,  y)  ®  fix  -  2x0,  y),  (1) 

where  2x<>  is  the  separation  between  the  two  input 
functions  on  the  input  plane  and  ®  denotes  the  corre¬ 
lation  operation.  Note  that  the  last  two  terms  repre¬ 
sent  the  cross-correlation  functions  that  are  diffracted 
around  (±2x0, 0).  Owing  to  the  limited  dynamic  range 
of  the  square-law  detector,  the  encoded  joint-trans¬ 
form  power  spectrum  is  usually  confined  to  a  small 
area.  Many  of  the  commercially  available  square-law 
detectors  used  in  JTC  experiments  have  a  much  larger 
recording  area,  and  therefore  the  detecting  devices  are 
not  fully  utilized.  If  the  collimated  readout  light 
beam  is  reduced  into  such  a  smc1’  "n  3-sectional  area, 
the  effective  width  of  the  cor:  neak  would  be 

increased  owing  to  convolution  oc  ween  the  correla¬ 
tion  function  and  the  focal  spot  of  the  readout  beam. 
Therefore,  to  determine  the  center  of  a  correlation 
spot  accurately,  a  readout  beam  of  large  cross  section 
is  preferable.  Moreover,  under  some  circumstances 
the  readout  light  cannot  be  reduced  into  such  a  small 
cross  section,  e.g.,  in  a  single  spatial-light-modulator 
JTC  architecture  in  which  the  collimated  light  is  used 
for  both  writing  and  reading.  Readout  light  power 
that  falls  onto  the  unencoded  area  of  the  square-law 
detector  would  not  be  utilized,  and  the  efficiency  of 
the  JTC  system  would  be  reduced. 

In  this  Letter  an  image  sampling  technique  is  pro¬ 


posed  to  increase  the  efficiency  for  a  given  JTC  sys¬ 
tem.  We  assume  that  a  one-dimensional  sampling 
grating  has  been  synthesized.  The  corresponding  am¬ 
plitude  transmittance  function  is  given  by 

CD  CO 

G(x)  =  ^  red/*  Kn  expijnr^), 

n“— <*>  '  '  n=— ® 

(2) 

where 

Jx  -  nd\  jl  lx  -  nd\  <  w!2 

reCt(~V  /  10  otherwise  '  <3) 

y  gsln(Wd),»^(n|p/<i)  (4) 

a  nzw/d  d 

w  and  d  are  the  pulse  width  and  the  period  of  the 
sampling  grating,  respectively,  and  r0  =  2ir/d.  Note 
that  the  Fourier  expansion  of  the  gratinr  is  given  as 
the  rightmost  expression  in  Eq.  (2). 

The  grating  is  then  inserted  at  the  input  plane  of  the 
JTC  (as  show  n  in  Fig.  1)  whose  input  functions  are 
assumed  to  be  two  identical  images  located  at  (x0,  0) 
and  (-x0,  0).  The  amplitude  transmittance  at  the 
input  plane  is  then  given  as 

30 

Tix,y)~  £  f Kn\fix,-x0,y) 

n--oo 

+  fix  +  x0,  y)]exp(/'rtrox)|.  (5) 

The  corresponding  joint-transform  power  spectral 
distribution  recorded  at  he  square-law  detector  is 

oo 

Sip,  q)  =  \  J\Tix,  y)jp  =  7  W;-\Fip-nrwq)? 

fits  — ns 

X  |1  +  cos[2x0(p  -  nr0)Ji,  (6) 
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Fig.  1.  High-efficiency  joint-transform  architecture. 


where  p  -  (2ir/f\)a  and  q  =  (2tt//X)/3  are  the  spatial 
frequency  coordinates,  F(p,  q )  is  the  Fourier  trans¬ 
form  of  fix,  y),  and  ?(•[  denotes  a  Fourier-transforma- 
tion  process.  To  ensure  that  the  spectra  do  not  over¬ 
lap,  it  is  assumed  that  the  sampling  rate  satisfies  the 
Nyquist  requirement,  i.e.,  d  <  ir/Ap,  where  Ap  is  the 
angular  bandwidth  of  the  input  object.  Thereiore, 
this  determines  the  number  of  orders  of  the  spectra 
that  can  be  written  onto  the  square-law  detector. 
Given  this  number,  to  obtain  almost  equal  intensity 
distribution  for  all  the  spectra  recorded,  the  width  of 
the  sine  function  smc(pw/2ir)  should  be  large  com¬ 
pared  with  the  dimension  of  the  detector  D,  i.e.,  D  < 
A/2oj.  Note  that  the  introduction  of  the  modulating 
term  Kn  in  Eq.  (6)  produces  power  spectra  of  different 
writing  intensities  that  enhance  the  dynamic  range  of 
the  square-law  detector. 

For  a  simplified  consideration,  let  us  use  only  the 
first  ±N  orders  of  diffraction  in  which  the  Kn  attenua¬ 
tion  factor  can  be  ignored,  i.e., 

N 

S{p,q)*  ^  2|F(p  -  nr0,  q)|2 

n*-N 


X  II  +  cos[2x0(p  -  nr0)]J.  (7) 


If  this  joint-transform  power  spectra  array  is  read  out 
with  a  coherent  source,  the  object  irradiance  at  the 
output  plane  becomes 


Ic(x,  y )  - 


N 


1  + 


2  cos inr^x) 


n=  1 


X  [12  fix,  y)  ®  f{x,  y)l2  +  I  fix,  y)  ®  fix  +  2x0,  y)|2 

+  I  fix,  y)  ®  fix  -  2x0,  y)|2].  (8) 


By  comparing  the  above  results  with  those  obtained 
from  a  conventional  JTC  [i.e.,  Eq.  (1)],  it  can  be  seen 
that  the  average  energy  of  the  correlation  function  in 
Eq.  (8)  is  increased  by 


dx  =  1+  2 N  times. 

(9) 


Howe.jr,  despite  the  increase  in  the  average  peak 
intensity,  the  correlation  function  is  modulated  by  the 
square  of  a  sinusoidal  factor,  and  this  might  not  be  a 
desirable  effect  in  some  cases. 


On  the  other  hand,  if  quasi-monochromatic,  partial¬ 
ly  coherent  light  is  utilized  as  the  readout  source, 
whose  coherence  width  equals  the  width  of  the  joint- 
transform  power  spectrum,  the  object  irradiance  at 
the  output  plane  can  be  written  as 

Ipcix,  y)  =  (1  +  2N)[\2fix,  y)  ®  fix,  y)|2  +  |/(x,  y) 

®  fix  +  2x0,  y)l2  +  l/(x,  y)  ®  fix  -  2x0,  y)|2].  (10) 

The  output  correlation  function  is  simply  amplified  by 
1  +  2 N  times  that  of  a  conventional  JTC.  There  is  no 
modulation  within  the  correlation  function,  and  maxi¬ 
mum  intensity  at  the  center  of  correlation  is  guaran¬ 
teed.  Moreover,  use  of  partially  coherent  readout 
light  would  also  suppress  coherent  artifact  noise  and 
thus  increases  the  signal-to-noise  ratio.  Note  that  the 
above  analysis  for  a  one-dimensional  sampling  grating 
can  be  easily  expanded  to  a  two-dimensional  sampling 
case.  The  spectra  array  given  in  Eq.  (4)  will  then 
become  a  two-dimensional  square  array  in  which  the 
multiple  orders  of  the  joint-transform  power  spectrum 
will  be  located  at  (nr0,  mr0).  Following  a  similar 
mathematical  analysis,  which  leads  to  Eq.  (10),  the 
correlation  spot  intensity  will  be  increased  by  a  factor 
of  (1  +  2 N)2  times  when  a  two-dimensional  sampling 
grating  is  used. 

To  verify  our  discussion,  an  experiment  using  a  2 f 
Fourier-transform  system  as  shown  in  Fig.  1  was  per¬ 
formed.  For  easy  implementation,  two  pieces  of  com¬ 
mercially  available  Ronchi  rulings  were  used  to  pro¬ 
duce  a  two-dimensiond  sampling  grid.  The  Fourier- 
transform  spectrum  of  the  grating  itself  shows  that  the 
±1  orders  are  approximately  half  the  intensity  of  the 


■# 


(a) 


(b) 

Fig.  2.  Experimental  results  of  the  JTC  when  no  sampling 
grating  was  used  on  the  input  plane:  (a)  joint-transform 
power  spectrum,  (b)  correlation  peak  obtained  by  using  a 
partially  coherent  readout  light. 
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(a) 


(b) 


Fig.  3.  Experimental  results  of  the  JTC  when  a  two-dimen¬ 
sional  sampling  grating  was  inserted:  (a)  the  3  X  3  power 
spectrum  array,  (b)  correlation  peaks. 

zero  order.  The  higher  orders  are  weak  and  to  permit 
an  easy  analysis  were  not  recorded.  The  input  pattern 
consists  of  two  identical  images  recorded  on  35-mm 
photographic  film.  A  He-Ne  laser  was  used  as  the 
coherent  light  source,  and  the  power  spectrum  was 
recorded  on  film.  Figures  2(a)  and  3(a)  show  the  pow¬ 
er  spectrum  recorded  before  and  after  the  sampling 
grating  was  inserted,  respectively.  Partially  coherent 
light  generated  from  a  white-light  source  in  conjunc¬ 
tion  with  an  interference  filter  was  then  used  to  read 
out  the  power  spectrum.  Figure  2(b)  shows  the  corre¬ 
lation  result  when  a  single  spectrum  order  was  used, 
and  Fig.  3(a)  shows  the  result  obtained  when  a  3  X  3 
spectra  array  was  used.  It  was  measured  that  the 
correia’ ion  peak  intensity  is  increased  by  approxi¬ 
mately  4.8  times  when  the  sampling  grating  is  inserted 
at  the  input.  However,  according  to  the  mathematical 


predication,  the  peak  energy  should  be  increased  by  9 
times.  We  believe  that  the  major  reason  for  the  de¬ 
crease  in  experimental  values  is  because  of  the  weak¬ 
ened  modulation  in  the  ±1  orders  of  diffraction  from 
the  Ronchi  rulings. 

In  conclusion,  we  have  proposed  and  demonstrated 
an  image  sampling  technique  that  improves  the  effi¬ 
ciency  for  a  given  JTC  system.  By  writing  multiple 
orders  of  the  joint-transform  spectrum  onto  the 
square-law  detector,  the  physical  size  of  the  device  as 
well  as  the  readout  light  power  are  more  fully  utilized. 
Consequently,  the  correlation  peak  intensity  is  sub¬ 
stantially  increased.  Under  either  coherent  or  par¬ 
tially  coherent  readout  light,  the  energy  of  the  correla¬ 
tion  function  thus  produced  is  proportional  to  the 
number  of  orders  of  spectra  used.  Such  merits  of  this 
proposed  technique  would  be  helpful  under  low  light 
intensity.  Moreover,  it  must  be  mentioned  that  the 
increase  in  the  correlation  peak  intensity  with  this 
technique  is  different  from  simply  increasing  the  read¬ 
out  light  power  while  reading  a  single  order  of  the 
joint-transform  spectrum.  When  random  noise  is 
added  in  during  the  JTC  process,  the  latter  approach 
would  increase  any  noise  spikes  present  on  the  output 
plane  by  the  same  amount  as  for  the  correlation  peak. 
However,  in  our  proposed  technique,  each  spectrum 
order  being  used  would  produce  different  noise  spikes 
at  different  positions.  Therefore  this  technique 
would  also  increase  the  discrimination  ability  of  the 
JTC  under  noisy  conditions. 

Note  that  if  a  charge-coupled-device  camera  con¬ 
nected  to  a  computer  is  used  as  the  square-law  detec¬ 
tor,  then  the  sampling  gratings  need  not  be  used  in  the 
optical  setup:  A  simple  digital  image-processing  algo¬ 
rithm  can  be  applied  to  replicate  a  single  order  of  the 
spectrum  into  multiple  orders  of  the  spectral  array. 
Since  the  JTC’s  can  be  regarded  as  generalized  optical 
processors,3  the  proposed  technique  can  improve  the 
performance  in  other  image-processing  applications. 
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Abstract.  A  hybrid  optical/digital  system  for  tracking  an  object  in  a  se¬ 
quence  of  images  is  described.  Since  a  joint  transform  correlator  does  not 
require  a  matched  spatial  filter  in  the  correlation  process,  object  tracking 
can  be  carried  out  by  continuously  updating  the  reference  image  with  the 
object  image  in  the  previous  frame.  This  adaptive  property  of  a  joint  trans¬ 
form  correlator,  together  with  the  parallelism  and  high  processing  speed 
of  an  optical  system,  ensure  high  correlation  between  objects  in  two  se¬ 
quential  frames.  The  relative  position  of  the  object  can  then  be  determined 
based  on  the  location  of  the  correlation  peak.  System  performance  is 
evaluated  and  experimental  demonstrations  are  presented. 
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1.  INTRODUCTION 

Optical  pattern  recognition  has  attracted  much  attention  in  var¬ 
ious  applications  because  of  its  parallel  processing  and  high¬ 
speed  operation  capabilities,  particularly  in  real-time  applica¬ 
tions  such  as  missiie  guidance,  vehicle  tracking,  and  automated 
lander  guidance  in  aerospace  missions.5  In  1974,  Vander  Lugt: 
demonstrated  the  use  of  matched  spatial  filters  (MSFs)  in  per¬ 
forming  cloud  motion  anaiysis  in  a  sequence  of  photographs 
taken  from  a  satellite  in  half-hour  intervals.  A  new  MSF  was 
made  for  every  photograph  in  order  to  correlate  with  the  scene 
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in  the  next  picture,  and  therefore  this  is  not  a  practical  real-time 
technique.  With  the  recent  development  of  various  kinds  of 
spatial  light  modulators  (SLMs),  many  real-time  tracking 
techniques  have  been  demonstrated.  Gara8  used  a  liquid  crystal 
light  valve  to  perform  real-time  tracking  based  on  a  fixed  MSF 
in  a  Vander  Lugt  correlator.  Using  a  liquid  crystal  television 
spatial  light  modulator,  a  dichromated  gelatin  multifocus  holo- 
lens  and  a  MSF  array,  Chao  and  Liu9  demonstrated  a  technique 
for  simultaneously  tracking  multiple  objects.  Optical  tracking 
using  novelty  filters  and  image  subtraction  have  also  been  pro¬ 
posed,1011  in  which  the  system  detects  the  difference  between 
sequential  image  frames  and  then  computes  the  position  of  the 
object  in  motion. 

The  joint  transform  correlator  (JTC)1  "13  is  an  alternative  ap¬ 
proach  to  optical  pattern  recognition.  It  has  been  demonstrated 
that  a  real-time  programmable  JTC  can  be  implemented  using 
two  SLMs14  or  just  a  single  liquid  crystal  television  (LCTV)  in 
a  video  feedback  fashion. 15-16  In  this  paper,  a  technique  for  real¬ 
time  adaptive  object  tracking  based  on  a  JTC  is  proposed  and 
demonstrated.  The  basic  idea  is  to  correlate  the  object  in  the 
current  frame  with  the  object  pattern  in  the  previous  frame  of 
sequential  video  images  so  as  to  obtain  the  relative  locations  of 
the  object  in  the  two  frames.  It  must  be  pointed  out  that  in  a 
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Fig.  1.  Optical  architecture  for  the  adaptive  JTC.  Cl,  C2  are  two  CCD 
cameras,  FL  is  a  48  in.  Fourier  transform  lens,  and  CL  is  a  50  mm 
camera  lens.  The  high-pass  filter  (HPF)  is  a  transparent  film  with  a 
tiny  blurred  dot  at  the  center. 


JTC  architecture,  the  reference  image  and  the  input  scene  to  be 
con-elated  are  placed  side  by  side  on  a  spatial  light  modulator. 
This  unique  property  of  a  joint  transform  correlator  enables  the 
system  to  adapt  to  the  input  scene  by  continuously  updating  the 
reference  image  with  the  previous  frame  to  correlate  with  the 
dynamic  object.  The  key  element  to  this  optical  JTC  is  a  high- 
contrast  LCTV,  and  the  hybrid  system  is  fully  controlled  by  a 
microcomputer. 

2.  OPTICAL  ARCHITECTURE  AND  THEORY 

To  improve  the  simplicity  and  compactness  of  the  system,  a 
single  LCTV  (Seiko  color  LCTV  model  LVD202,  220  x  240 
pixels)  together  with  a  Sony  CCD  camera  are  used  in  a  video 
feedback  architecture,  shown  in  Fig.  1.  Since  this  is  a  color 
LCTV.  three  neighboring  RGB  pixels  are  addressed  as  one  pixel 
unit  when  displaying  a  gray  level  image,  thus  decreasing  the 
resolution  of  the  device.  By  observing  its  Fourier  transform 
spectrum  (Fig.  2),  we  deduced  that  the  RGB  pixels  are  arranged 
in  the  fashion  shown  in  Fig.  3.  This  is  also  verified  by  observing 
the  pixel  structure  under  a  microscope.  The  effect  of  this  struc¬ 
ture  is  discussed  in  Sec.  4. 

To  interface  this  commercial  product  with  our  system,  the 
diffuser  and  the  protective  glass  of  the  LCTV  were  first  removed. 
To  compensate  for  the  phase  distortion,  the  liquid  crystal  display 
(LCD)  unit  was  then  separated  from  the  hardware  circuit  box 
and  immersed  in  a  liquid  gate  filled  with  mineral  oil.  The  two 
original  polarizers  attached  to  the  display  unit  were  left  un¬ 
touched.  A  half-wave  plate  was  inserted  in  front  of  the  He-Ne 
laser  to  align  the  plane  of  polarization  of  the  linear  polarized 
light  with  the  LCTV’s  front  polarizer.  In  this  airangement,  max¬ 
imum  transmission  and  a  higher  contrast  ratio  can  be  obtained. 

We  now  discuss  the  principle  of  the  tracking  algorithm.  Let 
us  assume  that  two  sequential  scenes  of  a  moving  object  are 
displayed  on  the  LCTV  via  the  computer's  video  frame  grabber. 
The  previous  frame  and  the  current  frame  are  positioned  in  the 
upper  and  lower  half  of  the  LCD.  respectively,  as  depicted  in 
Fig.  4.  Let 


/•- if-r  -  x»- 

/,(.<  -  *-i 


i,  y  -  y... 


-!) 


and 


ox.  v  -  vf-  i 


-s^i) 


be  the  image  functions  of  the  objects  displayed  on  the  LCD. 
where  2 a  is  the  height  of  the  display  unit,  t  and  /  —  1  arc  the 


Fig.  2.  FT  spectrum  of  the 
LCTV. 


Fig.  3.  Pixel  structure  of  the 
color  LCTV. 


Fig.  4.  Arrangement  of  the  two  frames  on  the  LCD.  2m  is  the  height 
of  the  display  unit. 


current  and  previous  frames,  and  (5r.5v)  is  the  relative  trans¬ 
lation  of  the  object  from  tr.e  U  -  I)th  to  the  rth  frame.  Then 
the  complex  light  field  at  the  frequency  plane  of  lens  FL  is  given 
by 

T(u.\)  =  F,- |fu.v)exp|  -i2t7  i«r-i-rv(y.-f-^~j  j  j 

u(x,.  i  +  6,r)  +  v^  v,  - 1  -r  by  -  ~  j  j  j  . 

(1) 


•f  Ft(u.  vlexps  -T2 


The  power  spectrum  recorded  on  CCD  camera  Cl  is  then  sent 
back  to  the  LCTV  for  an  inverse  Fourier  transformation.  Thus, 
the  complex  light  field  at  the  output  plane  becomes 

C(x.y)  =  i?-'{|7T  u.vjp} 

=  R,,{x.yi  +  R,- 1.;_  iix.y) 

-  R, j-i(x  -f-  bx.  \  -  by  -a) 

-  R,-tAx  -  bx.  >  -  ftv  a)  .  (2l 

where 

R„.„U.y)  =  .f  '{F„(u.viF*(u.v>} 

r*  f" 
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Fig.  5.  Experimental  setup  of  the  hybrid  system. 


Fig.  6.  Block  diagram  of  the  optical  digital  interface. 


is  the  correlation  function  between /m  and /„  and  .7  ’  '{-}  denotes 
the  inverse  Fojner  transform  of  the  function  {  }.  Note  that  the 
first  two  terms  of  Eq.  (2)  are  the  zero-order  terms  diffracted 
around  the  origin  of  the  output  plane,  while  the  last  two  terms 
represent  the  cross-correlation  terms  diffracted  at  .t^i  =  ax. 
Speak  I  =  (by  -  u)  and -tpcji;  =  -OX.  }pcix2_=  t  -  by  -  a). 
With  reference  to  our  previous  investigations.'  the  angular  and 
scale  tolerance  of  a  target  in  a  JTC  is  roughly  about  ~5r  and 
*  109c.  respectively.  By  assuming  that  the  motion  of  the  object 
is  relatively  slow  compared  to  the  processing  cycle  of  the  cor¬ 
relator./,-  i  will  strongly  correlate  with  f.  and  two  high-intensitv 
correlation  peaks  can  be  observed  at  the  output  plane.  The  new 
location  of  the  object  at  time  t  is  then  given  by 

C*  "  ■  i'  i  *  'sra-.;  -  —  ty.  §  *  -  in  t.'l 

In  our  experiment,  a  50  min  camera  lens  was  used  to  magnify 
the  Fi'uncr  spectrum  to  about  5  to  10  lines  mm  A  small  circular 
opaque  dot  wuh  a  blurred  edge,  which  acts  as  a  high  pass  filter, 
was  inserted  at  the  Fourier  plane  to  block  the  de  light  from  the 
CCD.  which  would  otherwise  cause  multiple  retleciions  .md 
scattering  at  the  surface  of  the  CCD  A  photograph  of  the  ex 
penmentai  setup  is  shown  in  Fsg.  5 

It  must  be  ment'oned  that  this  adaptive  correlation  technique 


Fig.  7.  Joint  transform  spectrum  recorded  by  02.  The  dc  light  is 
blocked  by  the  high-pass  filter. 


Fig.  8.  Correlation  result  from  Fig.  7.  One  of  the  two  peeks  is  shown 
at  the  top. 

requires  the  location  of  tnc  object  to  be  initialized  in  the  first 
frame.  This  can  be  done  using  the  same  joint  transform  corre¬ 
lation  system  but  with  a  prestored  reference  image  located  at  a 
fixed  position  on  the  LCTV.  In  actual  applications,  the  object 
might  have  distortion  variations  due  to  variations  in  size,  rotation 
orientation,  and  perspective.  Therefore,  a  hierarchy  search  of 
the  object  might  be  needed.  To  speed  up  the  searching  process, 
distonion  invariant  filters  such  as  the  circular  harmonic  expan¬ 
sion  filters  or  (he  synthetic  discriminant  function  filters  can  be 
employed. 

3.  OPTICAL  DIGITAL  INTERFACE 

The  hybrid  system  is  controlled  by  using  a  C  language  program, 
which  runs  on  an  IBM  AT  compatible  microcomputer  at  10 
MHz  epu  clock  speed.  Figure  6  is  a  block  diagram  of  the  digital 
system  hardware  configuration.  Two  video  frame  grabbers 
(IVGI2S  and  AT428  from  Daiacubc  Inc.),  which  can  digitize  a 
full  frame  of  input  video  signal  mm  a  5S4  x  5 1 2  image  array, 
have  been  installed  m  the  microcomputer.  The  video  input  o{ 
ihc  AT42S  hoard  is  connected  to  a  Fairchild  CCD  camera  (Cl  s. 
which  is  aimed  at  the  object  ot  interest.  The  video  output  o|  the 
AT42S  board  is  fed  to  the  LCTV  via  a  video  rf  signal  convener. 
CCD  camera  C2  is  connected  to  the  1VG12S  board  and  captures 
the  loini  transform  power  spectrum  during  the  first  half  o|  the 
processing  cycle.  Since  we  do  not  use  a  video  multiplexer,  the 
recorded  spectrum  is  sent  from  the  tvG  board  to  the  AT  board 
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Fig.  9.  Distribution  of  the  correlation  peaks  in  300  JTC  cycles. 


before  it  can  be  displayed  on  the  LCTV.  During  the  second  half¬ 
cycle.  the  correlation  result  is  captured  by  C2.  and  the  peak 
location  is  determined  by  sequentially  scanning  the  image  array 
for  the  pixel  with  maximum  intensity.  A  position  sensitive  de¬ 
tector  or  a  parallel  array  detector  would  definitely  increase  the 
detection  speed.  Despite  these  limitations  due  to  the  use  of  general- 
purpose  hardware,  our  system  runs  at  approximately  1 .2  s  per 
cycle.  It  is  possible  for  the  hybrid  system  to  run  at  half  the  video 
frame  frequency,  which  is  '/2x30  =  15  cycles  per  second,  if 
specialized  supporting  hardware  are  used. 

Moreover,  if  an  optically  addressed  spatial  light  modulator 
is  used  as  the  square  law  convener  instead  of  a  CCD  camera 
;  and  a  LCTV  used  in  video  feedback  fashion,  then  the  processing 
l  speed  could  run  at  video  frame  rate,  assuming  that  the  response 
time  of  the  second  spatial  light  modulator  is  much  shorter  than 
a  video  frame  cycle.  However,  the  power  consumption  and  size 

I  of  a  double  spatial  light  modulator  architecture  would  be  two 
times  higher  and  larger  than  that  of  this  proposed  single  spatial 
light  modulator  architecture. 

J4.  SYSTEM  PERFORMANCE 

To  see  the  performance  of  the  system.  CCD  camera  Cl  was  first 
focused  on  a  stationary  object.  The  JTC  then  correlated  two 

(identical  scenes.  The  joint  transform  spectrum  and  correlation 
output  captured  by  C2  are  shown  in  Figs.  7  and  8.  respectively. 
We  slightly  modified  the  program  so  that  the  original  image 
»displayed  is  locked  into  the  upper  half  of  the  LCD  and  the 
[updating  input  frame  is  always  displayed  on  the  iower  half.  The 
^program  was  then  set  to  run  for  300  cycles,  and  the  distribution 
of  the  correlation  peak  locations  is  plotted  in  Fig.  9.  The  jittery 
[video  signal  amplified  by  the  low-resolution  color  pixel  structure 
of  the  LCFV  might  be  one  of  the  main  sources  of  error  that 
cause  fluctuation  in  the  data. 


Fig.  10.  Correlation  peak  location  as  a  function  of  the  object's  trans¬ 
lation  in  the  x  direction.  The  object's  overall  translation  displayed 
on  a  185  mm  x  240  mm  monitor  screen  was  68.5  mm.  The  scale 
factor  was  computed  in  terms  of  the  distance  in  the  monitor  space 
per  pixel  unit 


Fig.  11.  Correlation  peak  location  as  a  function  of  the  object's  trans¬ 
lation  in  they  direction.  The  overall  translation  on  the  monitor  screen 
was  27.3  mm. 


Next,  we  translated  the  object  along  the  x  direction  from  0 
to  50  mm  in  5  mm  increments,  and  300  sample  data  of  the 
correlation  peak  locations  were  recorded  at  every'  step.  Figure  10 
shows  a  plot  of  (he  mean  of  each  of  the  data  sets  as  a  function 
of  the  object's  translation.  The  result  shows  an  excellent  linear 
relationship  between  the  object  space  and  the  correlation  space 
in  the  x  direction.  From  the  plot,  the  scale  factor  between  the 
two  coordinates  in  the  x  direction  can  be  evaluated  in  terms  of 
the  distance  in  the  object  space  (or  in  the  video  system's  space) 
per  pixel  unit. 

The  same  test  was  carried  out  in  the  y  direction,  and  the  result 
is  plotted  in  Fig.  1 1 .  An  overall  linear  relationship  is  observed, 
and  the  y  direction  scale  factor  is  3iso  evaluated.  It  is  interesting 
to  note  that  the  ratio  of  these  two  scale  factors. 

x  direciion  scale  facior  _  S.40I  ^  ^ 

v  direction  scale  factor  2.422 


should  be  the  same  as  the  inverse  of  (he  aspect  rano  of  Ihe  image 
array,  given  by1* 

i  monitor  screen  height  >-  No  horizontal  pixels 

aspect  ratio  monitor  screen  width  •  No  vertical  pixels 

which  is  a  measurement  of  the  pixel  element's  height  to  width 
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Fig.  12.  (a)  A  graphical  Image  revolve*  in  an  elliptical  path  at  21  s/rev.  The  dark  dots  are  the 
hand  track  of  the  tank's  positions  at  each  of  the  JTC  cycles,  (b)  Tracked  positions  of  the  tank 
after  one  revolution  and  (c)  after  four  revolutions. 


Fig.  13.  A  camera  on  a  space  vehicle  aimed  at  a  fixed  target  on  the 
ground  for  navigational  purposes. 

ratio  and  is  equal  to  0.578  in  our  video  system  (a  Panasonic 
B/W  monitor  in  its  under-scan-size  mode).  Comparing  this  num¬ 
ber  with  the  x,y  scale  factor  ratio,  the  two  values  are  different 
by  less  than  2%.  This  evaluation  can  serve  as  a  check  of  the 
system's  overall  accuracy. 

A  close  examination  of  Fig.  1 1  shows  a  periodic  stair-like 
data  structure  that  is  absent  in  Fig.  10.  and  we  have  also  recorded 
that  the  standard  deviations  at  the  “steps”  are  higher  than  usual. 
This  can  be  explained  as  follows:  When  the  object  is  translated 
in  the  x  direction,  the  joint  transform  fringes  are  generally  aligned 
at  an  angle.  When  the  object  is  translated  in  the  y  direction,  the 
fringes  are  aligned  in  the  x  direction.  A  study  of  the  pixel  struc¬ 
ture  in  Fig.  2  reveals  that  this  LCTV  is  better  able  to  display 
slanted  lines  and  vertical  lines  than  horizontal  lines.  The  coarse 
sampling  in  the  horizontal  direction  thus  generates  higher  data 
fluctuations  and  periodic  setbacks. 

After  these  preliminary  tests,  a  tank-shaped  graphical  image 
displayed  on  another  video  monitor  was  used  as  the  next  lest 
object.  The  tank  was  set  to  revolve  in  an  elliptical  path  at  a 
period  of  21  s/rev.  [Fig.  I2(a)|.  The  original  program,  which 
updates  both  halves  of  the  LCD,  was  set  to  run  autonomously. 
Figures  12(b)  and  12(c)  show  the  locations  of  the  tank  tracked 
by  the  system;  proper  x  and  y  direction  scale  factors  are  used 
in  these  plots.  Excellent  tracking  of  the  object’s  location  is  ob¬ 
tained  at  the  end  of  the  first  revolution.  However,  after  four 
revolutions,  some  deviations  of  the  tracked  points  from  the  cor¬ 


rect  locus  are  observed.  This  reveals  that  detection  error  accu¬ 
mulates  during  each  correlation  cycle,  as  can  also  be  seen  easily 
from  Eq.  (3).  This  drawback  of  an  adaptive  correlation  system 
must  be  considered  and  solved  by  using  some  other  method. 

5.  ADAPTIVE  CORRELATION 

A  major  merit  of  the  proposed  technique  is  the  adaptive  property 
of  the  system.  We  have  carefully  simulated  two  sets  of  image 
sequences  to  illustrate  this  concept.  The  first  set  simulates  a 
situation  in  which  a  camera  mounted  on  a  moving  vehicle,  say. 
a  space  vehicle,  is  imaged  onto  a  fixed  target  on  the  ground  for 
navigational  purposes,  as  shown  in  Fig.  13.  As  the  space  vehicle 
approaches  the  target,  the  detected  scenes  arc  continuously 
changing;  the  target  size  appears  larger,  and  its  orientation  and 
shape  also  change  due  to  different  perspective  angles  from  the 
vehicle  in  motion.  Using  computer-aided  design  graphics,  a  3-D 
tree-like  object  was  created  as  a  target  on  the  ground.  Nine  image 
sequences  simulating  exactly  the  same  scenes  as  captured  by  the 
moving  space  vehicle  were  recorded  from  a  CAD  terminal,  as 
shown  in  Fig.  14.  The  hybrid  tracking  system  has  little  difficulty 
correlating  the  targets  between  frames,  even  if  the  target  on  the 
first  frame  and  the  last  frame  look  very  different.  Figure  15 
shows  the  tracked  locations  of  the  target  as  seen  from  the  ve¬ 
hicle's  coordinate  frame. 

Another  simulation  illustrates  a  stationary  camera/moving  ob¬ 
ject  situation.  Figure  16  exhibits  the  motion  sequence  of  a  toy 
tank  moving  from  lower  left  to  upper  right  in  the  scene.  An 
overturned  truck,  which  serves  as  stationary  clutter,  was  also 
added  to  the  center  of  the  image  scene.  In  this  case,  four  objects 
are  displayed  on  the  LCD  during  the  correlation  process,  and 
eight  cross-correlation  functions  are  generated  on  the  output 
plane,  four  of  which  are  the  weak  cross-correlations  between 
the  tank  and  the  clutter  and  can  be  ignored.  Based  on  prior 
knowledge  that  the  clutter  is  stationary,  the  unwanted  peaks  arc 
always  located  at  (0 ,±a),  which  can  then  be  distinguished  from 
the  peaks  generated  by  the  target  in  motion.  Figure  17  shows 
the  joint  transform  spectrum  and  the  correlation  peaks  based  on 
frames  I  and  2  of  the  image  sequences.  The  tracked  positions 
of  the  target  are  given  in  Fig.  18. 

6.  CONCLUSIONS 

In  this  paper  we  have  demonstrated  the  use  of  an  adaptive  JTC 
for  object  tracking  based  on  the  correlation  between  images  in 
sequential  scenes.  The  high-speed  parallel  processing  ability  of 
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Fig.  14.  A  sequence  of  nine  images  were  recorded  from  a  CAD  terminal,  simulating  the  exact 
scenes  as  captured  by  a  camera  mounted  on  the  moving  space  vehicle.  Only  frames  1.  5,  and 
9  are  shown  here 


Fig  1b  Tracked  positions  of  the  ground  target  as  seen  from  the 
vehicle's  coordinate  frame 


& 


Fig.  16  Motion  sequence  of  a  toy  tank  and  a  stationary  truck  Six  frames  were  taken  front  a 
fixed  camera;  only  frames  2.  4.  and  6  are  shown  here  Note  that  in  frame  4.  the  tank  is  partially 
occluded  by  the  clutter 
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By  generating  a  blur  function  as  the  reference  object,  the  phase  of  the  spectrum  of  the  blur  function  is  automatically  canceled  in 
the  joint  transform  spectrum.  A  reference  function  is  then  utilized  to  compensate  the  amplitude  variation  cf  the  olur  spectrum  in 
order  to  restore  the  original  object.  Experimental  results  are  given. 


1.  Introduction 

The  jo;.'-  transform  correlator  (JTC)  was  pro¬ 
posed  by  Weaver  and  Goodman  [  1  ]  for  optically 
convolving  a  pair  of  2-D  functions  without  using  a 
complex  spatial  filter  in  the  frequency  plane.  This 
idea  was  further  applied  to  real-time  pattern  recog¬ 
nition  by  incorporating  some  spatial  light  modula¬ 
tors  in  the  optical  system  (2  j.  Recently,  deblurring 
using  a  nonlinear  JTC  was  suggested  [3].  In  this  pa¬ 
per.  we  demonstrate  experimentally  a  simple  method 
of  deblurring  using  a  joint  transform  processor  [4], 


2.  Joint  transform  processor 

Shown  in  fig.  1  is  a  joint  transform  processor  in 
which  the  configuration  of  the  three-lens  system  be¬ 
tween  the  source  point  s,  and  the  observation  plane 
P  is  a  standard  optical  information  processing  sys¬ 
tem.  P0  is  the  input  object  plane  and  Pi  is  the  joint 
transform  filter  (JTF)  plane.  Suppose  that  the  am¬ 
plitude  transmittance  of  the  input  object  is 

;^x,y)=at{x-x0,y)+a2(,x+x0,y) .  (1) 

After  square  law  detection  at  the  plane  P,,  the  am¬ 
plitude  transmittance  of  the  JTF  (or  joint  transform 
hologram )  can  be  written  as 

1  Y.S.  Cheng  is  jn  leave  from  the  institute  of  Optical  Sciences, 
National  Central  University,  Chungli,  Taiwan  32054,  R.O.C.. 


t,(fXJy)=\AdfxJy)\2+\AAL  C)|2 

i-2  P. e[Ai(fx,fy)  AUfxJy)  exp( -i4nx0fx)  ]  , 

(2) 

whereA,  is  the  Fourier  transform  of  a„fx-x/XF,  and 
fy=y/XF.  F  is  the  focal  length  of  all  the  lenses  L,  and 
X  is  the  wavelength  of  the  light.  Re  denotes  the  real 
part. 

In  the  read-out  process,  the  amplitude  transmit¬ 
tance  of  the  reference  function  A3(f„fy),  in  the  plane 
P2,  is  imaged  by  the  lens  pair  L3  and  L„  onto  the  JTF 
through  the  reflection  from  the  beam  splitter  BS. 
Light  reflected  from  the  JTF  is  collected  by  the  lens 
L2  and  the  inverse  Fourier  transform  of  A3(fx, 
fy)h(f*,fy)  occurs  at  the  observation  plane  P.  The 
output  amplitude  of  interest  is  either  IFT  [Ax(fx,  r) 
A\ (fx,  fy)  AyOc,  fy)  ]  or  IFT[.4?  (/„  fy)  A2(fx,  fy) 
As(,fx,fy)  1,  where  IFT  stands  for  the  inverse  Four.'  :r 
transform.  The  former  is  centered  at  x=2x0.  y=0 
while  the  latter  is  centered  at  x=-2x0,  y= 0.  In  gen¬ 
eral,  the  joint  transform  processor  can  ac!  ieve  what 
a  coherent  optical  processor  can  do.  However,  in  this 
paper,  we  would  concentrate  on  a  deblurring 
problem. 

Assume  that  the  amplitude  transmittance  of  the 
input  object  takes  the  form 

i(x,y)=o(x-x0,y)*recl[{x-x0)/l] 

+rect[(.t+.r0)//]  S(y)  ,  (3) 

where  the  first  time  term  is  the  blurred  object.  *  de¬ 
notes  convolution  operation,  and  /  is  the  blur  width. 
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P0  pi  P 


T*-  information  in  the  JTF  which  is  of  interest  to  us 
is 

MLJ-,)Al(fx,fy)=:0{fxJy)smc‘{lfx)  .  (4) 

where  the  unimportant  constants  are  neglected.  We 
note  that  the  phase  of  the  sine  function  is  completely 
canceled  due  to  the  presc  ’ce  of  the  reference  object 
rect(.v//)<J( y).  In  order  to  obtain  the  deblurred  im¬ 
age.  all  we  have  tc  do  is  to  generate  a  reference  func¬ 
tion  (sinc:(//<)  ]"'  which  cancels  the  undesired  am¬ 
plitude  variation  sine z(lfx)  in  the  object  spectrum. 
Thus,  the  output  wave  amplitude,  which  is  centered 
at  .v=2x0,  y=0,  is 

IFTM,  (fx,fv)  A'Afx,fy)  AyLf'j:,))  =o{x.  y)  . 

(5) 

If  programmable  real-time  spatial  light  modula¬ 
tors  (such  as  LCTV  are  placed  in  the  planes  P0  and 
P2,  then  the  references  object  A2  and  the  reference 
function  .-13  can  be  continuously  varied.  Or.  the  other 
hand,  a  real-time  square  law  detector  ( such  as  LCLV ) 


can  produce  the  JTF  instantly.  The  operation 
IFTt/Mt-J,]  can  then  be  achieved  in  real-time. 


3.  A  deblurring  system 

Although  blurring  can  occur  in  different  forms,  in 
this  paper,  we  only  consider  linear  blur.  In  order  to 
reduce  the  dynamic  range  requirement,  an  alterna¬ 
tive  deblurring  system  i”  which  Fourier  transfor¬ 
mation  takes  place  only  in  one  direction  is  suggested 
(fig.  2).  The  recording  system  and  the  read-out  sys¬ 
tem  can  be  put  together  as  those  in  fig.  1  using  a  beam 
combiner.  However,  for  the  ease  of  3-D  drawing,  we 
consider  them  separately.  Let  the  amplitude  trans¬ 
mittance  of  the  input  plane  be 

i  ( .V.  y  )=o(x-x0,  y )  *  reel  [  (.v-  xn )  //) 

+  rectj (a+.y0)//]  .  (6) 

Since  blurring  occurs  only  in  the  .v-direcuon.  we 
merely  have  to  perform  Fourier  transformations  in 
that  direction.  In  the  orthogonal  direction,  a  simple 
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4.  Experiment  and  discussion 

Shown  in  fig.  3a  is  the  original  object  in  which  the 
characters  ABDKU  serve  as  the  object.  The  thin  line 

a 


Fig.  2.  (a)  An  optical  system  for  recording  of  the  joint  transform 
spectrum,  (b)  A  read-out  optical  system. 

imaging  would  suffice.  Hence,  the  information  on 
the  blurred  object  and  the  reference  object  are 
brought  together  only  in  the  x-direction.  After  square 
law  detection  at  the  plane  P,,  the  JTF  takes  the  form 

h (/„  y)  - sinc2(//t )  [  1 0(/x,  y )  | 2  + 1 

+0(fx,y )  exp( -\<\nxofx) 

+0*Ux.y)exp(i47tx0fx)} ,  (7) 

where  0(fx,  y)  i ,  the  object  transformed  in  the  x- 
dircction. 

The  reference  function  [sinc2(^)  ]  - 1  on  the  plane 
P’  is  then  imaged  onto  the  JTF.  Finally,  an  optical 
system  (tig.  2b)  simiia*-  to  that  of  fig.  2a  brings  the 
light  to  the  output  plane.  Again,  inverse  Fourier 
transform  is  performed  in  the  x-direction  while  sim¬ 
ple  imaging  Is  maintained  in  the  y-direction.  The 
wave  amplitude  at  the  observation  plane  P  is 

k(x.  y) =o(x.  y)  *0(x.  y)+d'(x)  6(y) 

+o(x-2x0, y)  +«*(  _,v— 2x0, y) .  (8) 

We  note  that  the  third  term  of  the  above  equation  is 
the  deblurred  image.  For  an  amplitude  object,  the 
fourth  term  is  also  a  deblurred  image. 


D 
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Fig.  3.  (a)  The  original  object  ABDKU  with  a  reference  line. 


Fig.  3.  (b)  Blurred  object  and  reference  object. 


Fig.  3.  (c)  Joint  transform  spectrum. 
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Fig.  3.  'd)  Under-exposed  JTF  to  show  the  fringes  in  the  main- 
lobe  of  ihe  spectrum. 


e 


Fig.  3.  (e)  Overiy-exposed  JTF  to  show  the  fringes  in  the  side- 
lobes  of  the  spectrum. 

at  the  left  hand  side  of  the  figure  is  blurred  to  be  the 
reference  object.  By  photo  reduction,  both  the  char¬ 
acters  A3DK.U  and  the  reference  line  are  converted 
to  be  transparent.  The  photo-reduced  transparency 
is  then  placed  in  the  object  plane  of  r.  simple  imaging 
system.  Light  from  a  mercury  arc  source  shining  on 
a  piece  of  ground  glass  «erves  a  the  diffuse  light 
source.  Kodak  so-253  film  is  mounted  on  the  camera 


f 
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Fig.  3.  (f )  Reference  spectrum  I /sine 2(//t). 


rig.  3-  (g)  High-passed deblurrcd  image. 


above  a  digital  motorized  stepper.  The  motion  of  the 
camera  causes  the  recorded  image  to  be  blurred.  A 
further  contact  printing  makes  the  blurred  object 
transparent  again  (fig.  3b).  By  comparing  fig.  3a  with 
fig.  3b,  we  note  that,  since  the  degree  of  blur  is  cho¬ 
sen  to  be  very  severe,  the  blurred  image  is  beyond 
recognition.  The  distance  between  the  blurred  object 
and  the  reference  object  is  about  6  mm.  We  have 
chosen  the  original  line  to  have  a  finite  width  so  that 
the  reference  object  can  have  comparable  transmit¬ 
tance  as  that  of  the  blurred  object.  In  the  above  rc- 
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cording  processes,  linearity  is  ensured  by  pre-expos¬ 
ing  the  film  to  a  uniform  light  with  total  exposure 
somewhat  above  the  toe  of  the  H&D  curve. 

In  the  system  of  JTF  recording,  the  focal  length  of 
the  cylindrical  lens  CL,  is  30  cm  and  that  of  the 
spherical  lens  L|  is  20  cm.  Their  separation  is  ap¬ 
proximately  the  sum  of  their  focal  lengths.  Hence, 
the  image  is  compressed  in  the  y-direction  by  a  fac¬ 
tor  2/3.  Shown  in  fig.  3c  is  the  joint  transform  spec¬ 
trum  recorded  on  the  Kodak  649-F  plate.  Because 
we  have  exposed  the  blurred  object  in  the  linear  re¬ 
gion.  the  dark  region  of  fig.  3b  is  not  totally  opaque 
and  the  average  transmitted  light  is  squeezed  into  a 
vertical  line  by  the  Lns  L,  which  overly  exposes  the 
zero-frequency  spectrum.  Some  regions  in  the  spec¬ 
trum  are  also  overly  exposed  so  that  the  interference 
fringes  are  quite  lost.  To  reduce  the  effect  of  this 
problem,  more  light  from  the  reference  object  is  re¬ 
quired  in  order  to  faithfully  record  the  spectrum.  This 
can  be  done  by  placing  a  filter  in  front  of  the  char¬ 
acters  ABDKU  during  the  blurring  process.  In  fig. 
3d.  the  film  is  under-exposed  to  show  the  fringes  in 
the  mair.lobe  of  the  spectrum.  On  the  other  hand,  in 
fig.  3e.  the  film  is  overly  exposed  to  reveal  the  fringes 
in  the  sidelobes.  The  reference  function 
(sinc:(//,)  J-1  can  best  be  generated  by  computer. 
However,  in  the  experiment,  an  approximate  refer¬ 
ence  function  is  produced  by  exposing  a  649-F  plate 
to  the  spectrum  of  the  reference  object  only.  The 
negative  of  the  result  is  shown  in  fig.  3f  in  which  the 
nonuniformity  in  the  vertical  direction  is  caused  by 
that  in  the  reference  object. 

During  the  read-out  process,  the  reference  func¬ 
tion  is  imaged  by  a  lens  pair  L2  and  L?,  each  of  focal 
length  30  cm.  onto  the  JTF  (fig.  3c)  on  the  plane  P,. 
The  focai  length  of  both  the  cylindrical  lens  CL:  and 
the  spherical  lens  L4  is  30  cm.  Again,  they  arc  sep¬ 
arated  by  the  sum  of  their  focal  lengths.  Many  ref¬ 
erence  functions  and  joint  transform  filters  with  dif¬ 
ferent  exposures  and  different  chemical  processing 
conditions  arc  experimented.  The  image  of  the  orig¬ 
inal  object  which  shows  some  degree  of  dcblumng  is 
shown  in  fig.  3g.  We  note  that  the  image  is  stretched 
in  the  horizontal  direction  by  a  factor  of  9/4  wuh 
respect  to  the  vertical  direction  because  of  the  par¬ 
ticular  lenses  used.  Owing  to  the  presence  of  the  ver¬ 
tical  lines  in  both  the  joint  transform  filter  and  the 
spectrum  of  the  reference  function,  a  high-passed 
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image  is  expected.  However,  due  to  the  reason  that 
is  not  quite  clear,  the  edges  of  the  images  are  not  quite 
sharp.  The  horizontal  pans  of  all  the  characters  are 
quite  lost  because  of  overexposure  of  the  corre¬ 
sponding  regions  which  result  in  the  lost  of  the  in¬ 
terference  fringes.  This  problem  would  be  avoided  if 
a  continuous  tone  object  is  utilized  as  the  original 
object.  If  the  image  of  the  original  object  instead  of 
the  high-passed  object  is  expected,  the  unwanted  dark 
region  of  fig.  3b  should  be  painted  totally  opaque  be¬ 
fore  performing  the  read-out  process. 

Deblurring  is  mainly  a  combined  process  of  phase 
correction  and  high-pass  filtering.  Since  the  phase 
cancellation  is  automatically  achieved  in  the  JTF,  we 
merely  have  to  perform  a  high-pass  filtering  process. 
A  simpler  experiment  is  also  performed  in  which  the 
mainlobe  of  the  joint  transform  spectrum  is  overly 
exposed  to  generate  the  approximate  high-pass  func¬ 
tion  directly  on  the  JTF.  The  read-out  optical  system 
would  be  a  cylindrical  lens  together  with  a  spherical 
lens.  Comparable  result  as  fig.  3g  is  obtained. 


5.  Conclusion 

A  simple  deblurring  method  using  a  joint  trans¬ 
form  processor  is  suggested.  By  incorporating  some 
real-time  devices  into  the  optical  system,  the  process 
can  potentially  be  carried  out  in  real-time.  Although 
the  experiment  is  performed  with  photographic  films, 
the  result  nevertheless  shows  some  degree  of  de- 
blurring.  If  a  continuojs  tone  object  is  used  instead 
and  the  procedures  discussed  in  section  4  are  taken 
into  account,  the  deblurred  image  would  be  much 
better.  Some  other  methecs.  such  as  the  one  sug¬ 
gested  by  Yang  and  Leith  { 5 ),  can  also  be  utilized  to 
restore  the  original  object. 

Based  on  the  simple  method  we  have  suggested, 
for  a  general  blur  (dciocusing,  nonlinear  motion, 
etc.),  we  would  generate  the  same  blur  as  the  ref¬ 
erence  object.  The  phase  of  the  blur  spectrum  is  au¬ 
tomatically  canceled  in  the  joint  transform  spcc- 
Tum.  Then,  we  would  generate  a  reference  function 
to  cancel  the  nonumformiiy  of  the  blur  spectrum. 
Thus,  a  debluTcd  image  can  be  obtained. 
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ABSTRACT 

In  this  paper  tee  shail  show  that  blurred  photographic  images  due  to 
linear  mot  ton  can  he  restored  tilth  a  lotrtt  transform  processor  iJTPl 
The  deconvolution  function  for  the  image  dehlurnng  can  ne  directly 
implemented  at  the  input  plane  of  a  JTP.  The  btpoiamv  of  the  deconvo¬ 
lution  function  can  ne  accomplished  hv  encoding  with  a  rt-shified  grating 
structure.  Computer  simulations  to  test  the  feasibility  ui  the  scheme  are 
given. 

One  of  the  major  advantages  of  a  joint  transform  architecture 
(1.  2]  must  be  the  avoidance  of  the  filter  synthesis,  and  the 
architecture  of  the  processor  can  also  be  applied  to  general¬ 
ized  image  processing  (31-  Research  in  ibis  area  has  been 
motivated  by  recent  advances  in  spatial  light  modulator  tech¬ 
nology  and  the  inherent  reai-time  processing  capability. 

This  paper  demonstrates  the  implementation  of  a  joint 
transform  processor  (JTPl  architecture  to  restore  blurred  ;m- 
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Figure  1  A  one-dimensional  joint  transform  processor 


ages  due  to  linear  mouon.  In  a  conventional  coherent  optical 
processor,  image  deblurring  can  be  performed  by  synth esizine 
a  restoration  filter,  whose  transfer  function  is  the  inverse  of 
the  blurred  function  [4].  Notice  that  the  inverse  filter  is 
generally  not  physically  realizable  due  to  manv  concomitant 
difficulties.  In  a  JTP.  however,  the  object  is  made  to  convolve 
with  a  processing  (reference)  function  that  is  placed  at  the 
input  piane.  In  this  arrangement,  some  of  the  processing 
constraints  may  be  alleviated.  Because  the  deblurring  funcucn 
:s  generally  a  bipolar  function,  we  have  proposed  a  spatially 
encoding  technique  to  produce  the  bipolarity  of  the  processing 
junction,  in  other  words,  two  sets  of  graunz  of  the  same 
frequency  are  encoded  on  the  input  transparency  to  produce  a 
s'  phase  shifted  vector,  such  that  a  bipolar  nature  of  the 
processing  function  can  be  obtained  in  the  first-order 
diffraction. 

Figure  I  shows  a  1-D  joint  Fourier  transform  correlator, 
where  cylindrical  lens  L.  has  no  effect  in  the  x  direction. 
Thus,  the  Fourier  transforms  of  the  object  and  the  reference 
function  in  this  direction  are  obtained  at  P,  by  L..  In  the  r 
direction,  however,  the  input  functions  are  imaged  onto  P-;  by 
the  combination  of  the  two  lenses  L,  and  L-.  Thus,  a  i-D 
joint  transform  hologram  can  be  obtained  at  P}  with  a 
square-law  detector  tSLD).  We  assume  that  an  object  trans¬ 
parency  f.jx.  v)  and  a  reference  A(x.  y)  are  inserted  in  the 


input  plane,  t  ne  transmittance  distnbuuon  can  be  written  as  87 

f(.r.  v )  «/,(.x  -  a.  v)  -  A ( x  -  a.  v)  (l) 

where  2a  is  the  separation  of  A  and  A.  By  coherent  tliumina- 
f.on.  the  compiex  light  distnbuuon  at  the  Founer  piane  a 
given  by 

T(  p)  =  F,(  p) expt  -:a p)  -  Fz(  p)exp(iap)  (2) 

where  p  represents  the  angular  spatial  frequency  coordinate. 

*(  P )  =  F{  /,( x. FAp )  =  F{  /,( x.  >•)}  (3) 

and  F  denotes  a  1-D  Fourier  operator.  The  transmittance  of 
the  joint  transform  hdogram  produced  by  a  square-law  detec¬ 
tor  can  be  wntten  as 

\T(p)y  =  <F.\y  -  ‘F-i1  -  F,F*  ex d(  -lap)  -  F*F-ex p(2ap) 

(4) 

The  system  setup  tor  the  readout  process  is  the  same  as 
shown  in  Figure  i.  By  placing  the  joint  transform  hologram  at 
the  input  piane.  the  cross  correlation  of  functions  A  and  /. 
can  be  observed  at  (2<z.0)  and  (  -  2a. 0)  of  the  output  plane.  If 
one  of  the  two  functions  is  flipped  along  the  x  direction,  for 
example.  /,(x  -  a.  y  j.  then  the  convolution  operauon  instead 
of  the  correlation  can  be  performed.  It  is  therefore  evident 
that  JTP  can  be  used  as  a  generalized  image  processor. 

Let  us  assume  that  the  blurred  function  is 

.?( x)  °  ot  x)  *  rectj  —  j  (5) 

where  o(x)  represents  the  unbiurred  object. 

rectff)- P*  -  A/2  £  x  <  L/2 
\  LI  1 0.  otherwise 

is  the  linear  smearing  function.  A  is  the  smearing  length,  ami 
•  denotes  the  convolution  operauon.  it  is  apparent  that,  for 
general  image  processing,  the  output  compiex  tight  field  can 
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Figure  2  The  wuse  m  she  proeesur.g  function.  j;vcn  u  -  ■  i  L  -  a  -  •«  I 
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(a) 

Figure  4  Grating  encoding  pattern  applied  to  the  processing  func- 
U  00 

."(x)»recUx/Li 

-3L.  .III. 

Figure  3  .'2i  An  cttsssie  of  modified  processing  fisneeom  ibi  con¬ 
volution  of  is!  with  the  smeanet  function 


be  written  as 

,-fx)  -  g(x)«6(xj  =  <>(x)*rect(-|»6fx)  (6) 

where  h(x)  is  ihe  spauai  impulse  response  or  the  niter  func¬ 
tion.  it  caa  be  shewn  that,  for  a  unear  motion  intake  cebiur- 
rin*.  the  spatial  impulse  response  of  a  debiumng  niter  ts  [see 

the  appendix! 


hi  x)  =  k- 


sin(  <uixj)sinhf  ut) 

( cost  ux  I  -  cosbfwatj' 


where  *  is  a  proportionality  constant  and  a  is  as  arbtfraniy 
stnail  positive  parameter  that  determines  the  width  of  the 
spikes  of  hi  r  ■.  as  shown  in  Figure  2.  However, nracuce.  the 
debiumng  junction  six)  is  spatially  Hunted.  and  the  spikes 
are  reshaped  into  rectangular  form  for  opucai  implementa¬ 
tion.  as  used  by  Swindell  [5]  and  shown  sn  Figure  ifa).  The 
corresponding  restored  point  image  is  deptaed  in  Figure  itb». 
To  separate  the  image  from  the  ghost  images,  it  is  required 
that  the  sire  of  the  reference  should  be  larger  than  double  the 
object  sac.  :.e..  L-.  >  IL. 

The  btpoiamy  of  the  debiumng  function  h-.x'i  can  be 
obtained  by  encoding  a  set  of  v-shifted  Ronchi  gratings  >61.  as 
shown  m  Figure  4  The  transmittance  of  the  encoded  trans- 
parency  caa  be  wntten  as 

*  X 

T,(x.  vi  =  h.i  x)  £  of  v  -  lbk)rccU  y/fcl 

*  —  “  X 

*  * 

-  h-i  x  l  T'  jf  «•  _  2bk  -  oStecU  r  b)  f  SI 


Figure  S  fa)  A  binned  image  dae  to  linear  moeoe:  ib)  output 
debfemed  usage  from  ta) 


where  26  is  the  grating  period,  ana  ft.(x)  and  hzix)  are  the 
paative  and  negative  functions  of  k(xt.  By  i-D  Fourier 
transforming  the  above  equation  with  respect  to  the  y  axis,  it 
becomes 


T,(x.c)  =  V  S(q  -  kir/b)[k.{x)  -  expf /«)*;( x)j 


where  a  ts  the  spauai  frequency  coordinate.  The  nm-order 
deblurred  image  can  be  obtained  from  the  joint  transform 
hologram  as  expressed  in  Equauon  (6).  It  is  therefore  appar¬ 
ent  that  the  image  debiumng  caa  be  obtained  with  the  pro¬ 
posed  i-D  JTP. 

Tim  feasibility  of  the  optical  unpkaeotaiica  is  studied  by 
a  computer  simulation  as  given  in  Figure  5.  Figure  ifai  shows 
a  blurred  image  with  a  smearing  length  L  =  10  pixels.  The 
rectangular  spikes  of  the  debiumng  function  dtxi  are  I  pixel 
wide.  The  corresponding  computer-stmulated  resuii  is  shown 
in  Figure  itb). 

Finally.  «e  wouid  condude  that  restoring  blurred  image 
using  a  one-dimenssonai  joint  transform  processor  is  feasible. 
The  bipolarity  of  the  processing  function  can  be  alleviated 
with  a  grating  encoding  scheme.  This  svsietn.  in  pnncpic.  can 
be  imnlemented  with  real- ume  devices. 
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APPENDIX 

The  spauai  impulse  response  function  h(x)  can  be  expressed 
by  the  Cauchy  pnndpie  vaiue  with  the  inverse  Fooner  trans¬ 
form  of  an  inverse  filter,  such  as 


h(x) 


I  extH-®pj)|sina  -}|  expljxs  idv 

--■*  L  \w/j  ’  ‘  ' 

2/  exp(-ap)  SincJ -f  casixp)  dc  (10) 
-o  l  lutj 
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where  a  —  0.  A  dosed  form  of  Afx)  can  be  derived,  based  on 
the  sum  of  two  contour  integrations,  given  by 

1  “  Jc -«i[sM  ~  l]  «P<  jxz)  dr 

-  exp( ar)|snd  —  j  j  exp(  ixz \  dz  { 11) 
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where  contours  C  and  C  are  shown  in  Figure  6.  When 
x  >  0.  /  =*  0  and  it  can  be  shown  that 


h(x) 


X 

—  lx*  y,  {-l)m  txp(-aku\siia(  kwxl 

i-l 

sum  wx  isinhi  wai 

=  <**  - 

ICOSl  «JC|  -r  coshf  uidl}* 


(12) 


When  x  <  0.  hixi  *  ofjxi)  because  Afjci  is  an  eves  function. 
Thus,  the  general  form  of  k(xh  as  ta  Equation  (7).  can  be 
obtained. 
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photorefractive  holograms 
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Using  coupled  w*ve  theory  and  the  I**-  of  retraction-  diffraction  properties  of  voiunre  hoku trams  are 
discussed.  Reconfigurable  interconnections  by  either  wavelength  tuning  or  spatial  division  techniques  are 
proposed-  Reflection  type  volume  holograms  can  be  used  for  a  large  number  of  reconfizvrtibie  interconnec¬ 
tions  in  terms  of  finite  wavelength  tupabtiity.  Transmission  volume  holograms  encoded  in  pinhole  hoio- 
graias  can  be  easily  reconfigured  by  spatiai  light  modulator.  Experimental  demonstrations  ohtiiE*d  by  using 
these  methoos  are  presen'ed- 


L  Modbcticn 

In  recent  years,  high  speed  communicat.  a  and  high 
speed  computation  ha'  e  aroused  interest  in  optical 
interconnections. *  The  advantages  of  optical  inter¬ 
connections  over  electrical  interconnections  are  high 
speed,  massive  parallelism,  and  lower  mutual  interfer¬ 
ence.  Optical  fibers,  integrated  waveguides,  and  free- 
space  interconnections  are  all  being  considered-  How¬ 
ever,  tree-space  interconnection  is  the  most  attractive 
among  the  others  for  its  potential  massive  parallelism. 
Both  classical  and  holographic  optical  elements  are 
being  considered  as  candidates  for  free -space  intercon¬ 
nections.  Generally,  classical  optical  elements  have 
higher  efficiency  of  light  utilization:  nevertheless  the 
holographic  optical  elements  have  greater  flexibility. 

Reconfigurable  massive  interconnection  is  impor¬ 
tant  for  communication  and  computation  aspects. 
Spatial  light  modulators  CSLMs)  were  proposed  for  use 
as  a  reconfigurable  optical  crossbar3  based  on  optical 
matrix-vector  multiplication.  The  main  problem 
with  the  8LM  method  is  low  energy  efficiency-.  In 
addition,  the  SLMs  suffer  from  low  contrast,  low  reso¬ 
lution.  slow  response,  and  Urge  physical  size.  To  in¬ 
crease  the  energy  efficiency,  Yeh  ei  ai.s  proposed  a 
two-wave  mixing  technique  to  amplify  the  controlling 
pattern,  which  is  generated  by  the  SLM.  However, 
high  light  intensity  (—1  W/cm r)  is  required  for  high 
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efficiency  of  energy  transference,  and  the  method  may 
not  be  applicable  at  the  present  time. 

In  recent  years,  laser  technology  has  rapidly  ad¬ 
vanced  with  compact  wavelength  tunable  laser  diodes 
becoming  available-  Their  wavelength  tunability  pro¬ 
vides  a  convenient  method  of  controlling  the  direction 
of  diffracted  light  beams.  Lur  has  proposed  a  tech¬ 
nique  for  using  a  multiplexed  waveguide  transmission 
hologram  and  wavelength  tunable  light  source  to  real¬ 
ize  the  reconfigurable  optical  interconnections.  Gen¬ 
erally  speaking,  the  wavelength  tunable  range  AA/X  of 
laser  diodes  is  ~i0“:.5  However,  for  transmission  ho¬ 
lograms.  10--  relative  wavelength  variation  would  pro¬ 
duce  a  very  small  deflection  of  the  laser  beam.  On  the 
other  hand,  with  a  surface  waveguide,  the  usable  space 
is  essentially  one-dimensional.  These  are  the  major 
reasons  that  limit  the  number  of  interconnections  us¬ 
ing  transmission  holograms  and  surface  waveguides. 
Multifacet  computer-generated  holograms  may  also 
1*  used  for  reconfigurable  optical  interconnections. 
However,  for  high  fanout  interconnections,  the  energy- 
efficiency  of  the  holograms  is  generally  low 

It  is  well  known  that  a  photorefractive  volume  holo¬ 
gram  can  have  high  diffraction  efficiency  and  high 
angular  and  wavelength  seiectivities.  We  first  discuss 
the  performance  of  transmission  and  reflection  holo¬ 
grams  based  on  coupled  wave  theory  and  law  of  refrac¬ 
tion.  Then  reconfigurable  optical  interconnection 
schemes  using  wavelength  tuning  and  spatial  division 
techniques  are  proposed,  A  small  amount  of  wave¬ 
length  tuning  can  be  used  to  realize  a  large  number  of 
reconfigurable  interconnections,  which  can  be  ob¬ 
tained  by  choosing  the  wriiing  angle  and  thickness  of 
the  hologram.  For  the  spatial  division  method,  the 
interconnection  patterns  can  be  recorded  as  a  series  of 
pinhoie  holograms  in  the  photorefractive  material. 
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This  series  of  pinhole  holograms  can  be  reconstruct¬ 
ed  with  an  array  of  light  sources  and  by  simply  trans¬ 
lating  a  pinhole  generated  by  an  SLM.  and  the  inter¬ 
connection  pattern  can  be  reconfigurated.  We  shait 
experimentally  demonstrate  the  reconfigurabilities  of 
both  techniques  using  a  nonlinear  photorefractive 
crystal,  LiNbO*. 


I.  Properties  of  Volume  Holograms 

We  first  discuss  some  basic  properties  of  volume 
holograms.  We  assume  that  two  write-in  beams  form¬ 
ing  an  angle  26  are  entering  into  a  photorefractive 
material  as  shown  in  Fig.  1.  The  vector  magnitude  of 
the  interference  grating  within  tbs  crystal  can  be  writ¬ 
ten  as 

2s  six*) 

Ms  *  .  ti» 

where  X  is  the  writing  wavelength  and  n  is  the  average 
refractive  index  of  the  crystal  If  the  readout  wave¬ 
length  is  deviated  from  X  bv  XX.  the  Bragg  diffraction 
condition  requires  a  change  of  A6.  By  differentiating 
Eq.  (1)  with  respect  to  X  and  6.  respectively,  the  rela¬ 
tionship  between  XX  and  A6  is  given  by 

—  =  -  I  aoh  m 

>  2 


for  which  the  change  of  X  should  be  compensated  by 
the  change  of  6.  We  call  26  the  internal  writing  angle 
within  the  crystal.  Note  that  the  external  writing 
angle  2a  is  different  from  the  internal  writing  angle  28. 
By  applying  the  law  of  refraction  to  Bq.  (2)  for  the 
unsianted  transmission  holograms  in  Fig.  1  (a),  we  have 


-10  =  - 


\.r  — 


-Ul  _  _  __  I  csar* 

— ■[  =  —  —  — - 7“ 
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€4* 


Similarly  as  for  the  unsianted  reflection  hologram  in 
Fig.  Kb),  we  have 


i0> 


where  2a  is  the  external  write-in  angle.  We  note  that 
only  the  second-order  term  in  Eq.  (8)  is  significant  as  a 
approaches  90*.  However,  we  shall  consider  cases  for  a 
in  the  10-80°  region,  and  all  second-order  terms  can  he 
neglected.  The  reciprocal  of  the  dispersions 

/.  i  dx\ 

{ <-«■-  t  j- ! 

for  transmission  and  reflection  holograms  are  func¬ 
tions  of  external  write-in  sngie  a  in  Eqs.  (4)  and  (6)  and 
are  shown  in  Figs.  2(a)  and  (6).  We  see  that  under  the 
same  write-in  angle  2a.  the  reflection  hologram  would 
provide  higher  dispersion,  which  is  due  to  the  refrac¬ 
tion  at  the  surface  of  the  photorefractive  crystal 
Since  the  refractive  index  of  the  crystal  is  higher  ihan 


(a)  (b) 

Fir.  I  , .j aage aside  oatalde  the  rtccrd-cg  rntdix.  t*i 
j  -  r.iistsr.  bskgnes:  I b)  re'.«'ar.  htApote. 


SO! - - —  - - - 

c  «S  K«s*»nNfs 

a 

F%.2-  DapenBee<L/XH<£A/da)>saft5*cwec# fcaa-wrisisgastse 
je.s  =  2Xfc  e.  iasaatja;  nikcas 


the  index  of  air.  the  internal  angle  6  is  less  than  the 
external  angle  a  for  the  transmission  hologram  and  G  > 
a  for  the  reflection  hologram.  This  means  that  for  the 
same  external  angle  a.  the  reflection  hologram  would 
have  higher  dispersion.  On  the  other  hand,  when  a 
approaches  0*.  the  angular  magnification  CA«/A©i  be¬ 
comes  larger  due  to  refraction,  as  can  he  deduced  from 
Eq.  to).  This  would  also  cause  higher  dispersion  in 
reflection  hokfrarrs.  In  the  region  between  a  =  30 
and  60s.  a  uniform  dispersion  is  observed  for  the  re¬ 
flection  hologram. 

To  analyze  the  angular  and  wavelength  seieeUvities 
of  volume  holograms,  we  shall  refer  to  the  coupled 
wave  theory  obtained  by  Kogelnik.4  Let  us  assume 
that  the  multiple  diffraction  process  can  he  neglected 
for  a  muluexpasure  hologram.  The  problem  can  he 
treated  with  the  weak  coupling  approximation.  Thus, 
by  referring  to  the  coupled  wave  theory,  the  normal¬ 
ized  diffraction  efficiency  for  an  unsianted  transmis¬ 
sion  hologram  car.  he  expressed  as 

<Ti 

where  r  -  rn,d/X  cos#.  n%  is  the  amplitude  of  the 
spatial  modulation  of  the  refractive  index,  d  is  the 
thickness  of  the  hologram. 
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2jrnd  sin#  ,  „ 
- - - A9 


2 xnd  sin~8  AX 
X  cos©  X 


is  known  as  the  dephasing  parameter,  which  Is  depen¬ 
dent  on  the  reading  angular  deviation  A0  and  the 
reading  wavelength  deviation  (AX/X). 

For  the  weak  coupling  approximation  (i.e.,  H  «  N  )> 
Eq.  (7)  can  be  simplified,  as  given  by 

ij  =  k2  sinc2f.  (8) 


The  angular  and  wavelength  selectivities  of  the  holo¬ 
grams  can  be  defined  by  the  full  width  of  the  main  lobe 
of  the  sine  function  and  are  given  by 


|A0|(  = 


X 

nd  sin© 
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Fig.  3.  Angular  selectivity  of  volume  holograms  normalized  by  X/d, 
n  =  2.28:  a,  transmission  hologram;  6,  reflection  hologram. 


where  subscript  t  signifies  transmission  hologram. 

With  reference  to  Eq.  (3)  and  the  law  of  refraction, 
we  find  that 


|A“li  = 


2  7 

yrr  —  sura 
sina  cosa 


X 

d  ’ 


(11) 
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(12) 


Similarly,  for  an  unslanted  reflection  hologram,  the 
normalized  diffraction  efficiency  can  be  expressed  as 

1  (13) 


where 


1  +  {l 


k  =  jrnxd!\  sin0, 
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Fig.  4.  Wavelength  selectivity  of  volume  holograms  normalized  by 
\/d,  n  =  2,28:  o,  transmission  hologram  (the  scale  on  right);  6. 
reflection  hologram  (the  scale  on  the  left). 


For  the  weak  coupling  case  (i.e.,  |  «|fj),Eq.  (13)  can 
bi-  reduced  to  the  following  expression: 

7J  —  1 H2  sinc2f-  (14) 

The  equations  for  angular  and  wavelength  selectivities 
of  an  unslanted  reflection  hologram  can  be  derived, 
such  as 


|A«ir  = 

vn2  -  cos -a 

X 

(15) 

sina  cosa 

d 

2 

i 

X 

(16) 

U  Jr 

\n2  -  cos'a 

d  ’ 

where  subscript  r  signifies  the  reflection  hologram. 

Figure  3  shows  the  angular  selectivities,  (Aa)(  and 
( Aa)r ,  as  functions  of  an  external  write-in  angle  a, 
where  the  values  of  Aa  have  been  normalized  by  X/d. 


From  this  figure  we  see  that  the  optimal  values  occur 
at 

(a,)opt  =  arcsinv'n(n  -  vt2  -  1),  (17) 

(ar)„pl  =  arccosy  n(n  -  v n-  -  1 ),  (18) 

respectively.  For  example,  if  n  =  2.2P  for  LiNb03 
crystal,  (af)opt  -  46.5°  and  («r)opt  85  43.5°.  We  note 
further  that  in  the  region  of  a  =  30-60°,  the  hologram 
has  the  sharpest  angular  selectivity  both  for  transmis¬ 
sion  and  reflection  holograms. 

Figure  4  shows  the  wavelength  selectivities 


as  functions  of  a.  From  this  figure  we  see  that  the 
wavelength  selectivity  for  the  transmission  hologram 
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(c)  (d) 

Fig.  5.  Diffracted  wave  vectors  derived  according  to  momentum 
conservation  (the  horizontal  components  of  the  dephasing  vector 
should  always  be  zero):  (a)  transmission  hologram  with  incident 
angular  deviation;  (b)  reflection  hologram  with  incident  angular 
deviation;  (c)  transmission  hologram  with  reading  wavelength  devi¬ 
ation;  (d)  reflection  holoram  with  reading  wavelength  deviation. 


Fig.  6.  Principle  of  reconfigurable  optical  interconnects  with  vol¬ 
ume  holograms.  OA  and  OB  are  two  grating  vectors  with  the  same 
magnitude  and  an  angular  separation  A9.  CE  and  DF  are  the 
perpendicular  bisectors  of  OA  and  OB. 


increases  as  a  increases.  However,  transmission  holo¬ 
grams  generally  have  a  poorer  wavelength  selectivity 
than  the  reflection  hologram.  The  wavelength  selec¬ 
tivity  of  the  reflection  hologram  is  relatively  unform 
over  a,  which  is  ~0.45 X/d. 

With  reference  to  the  principle  of  momentum  con¬ 
servation,  the  diffracted  wave  vectors  can  be  deter¬ 
mined.  Let  us  assume  that  the  holographic  recording 
medium  is  infinitely  extended  in  the  x-  and  y-direc- 
tions  and  has  a  finite  thickness  d  in  the  z-direction.  A 
dephasing  wave  vector  can  be  defined  as 

Alt  =  k|  -  k2  -  ki  +  kj,  (19) 

where  ki  and  ko  are  the  two  writing  wave  vectors,  k*  is 
the  reading  wave  vector,  and  k*  is  the  diffracted  wave 


vector.  In  terms  of  momentum  conservation,  it  is 
required  that 

(Afe),  =  (A/sh  =  0.  (20) 

Thus  the  z-component  of  Ak  is  directly  related  to  the 
dephasing  parameter  f  in  Eq.  (7)  by 

r=(A k).-d.  (2D 

Equation  (7)  can  be  used  to  determine  the  diffracted 
wave  vector  as  shown  in  Fig.  5.  For  simplicity,  we  have 
assumed  that  all  the  wave  vectors  are  in  the  x-z  plane. 
Figure  5(a)  shows  that  the  reading  angle  of  the  trans¬ 
mission  hologram  deviates  the  amount  A0  from  the 
write-in  angle  0.  By  referring  to  Eq.  (20) ,  the  diffract¬ 
ed  wave  vector  should  deviate  -A0  in  the  first-order 
approximation.  The  wave  vector  diagram  for  the  re¬ 
flection  hologram  is  shown  in  Fig.  5(b).  In  this  figure 
we  see  that  the  angular  deviations  of  the  reading  and 
writing  wave  vectors  are  identical.  Figures  5(c)  and 
(d)  show  the  effects  due  to  reading  wavelength  dephas¬ 
ing.  For  the  transmission  hologram  [i.e.,  Fig.  5(c)],  the 
diffracted  wave  vector  deviates  by  A0;  for  the  first- 
order  approximation,  we  have 

A0  =  2  tan0  •  (22) 

A 

This  indicates  that  for  a  finite  wavelength  bandwidth, 
it  will  cause  the  diffracted  focal  spot  to  smear.  How¬ 
ever,  reflection  holograms  do  not  suffer  from  such  focal 
spot  smearing,  since  the  wave  vectors  for  unslanted 
reflection  holograms  are  always  symmetrical  about  the 
x-axis. 


III.  Reconfigurability  with  Wavelength  Tuning 

The  principle  of  the  reconfigurable  interconnection 
using  wavelength  tuning  can  be  seen  as  illustrated  in 
Fig.  6.  The  grating  vectors  OA  and  OB  are  written  in 
the  photorefractive  crystal  with  an  angular  separation 
A0.  When  the  grating  OA  is  read  out  with  wave  vector 
OC,  the  diffracted  wave  vector  is  denoted  by  AC  (CE  is 
the  perpendicular  bisector  of  OA).  On  the  other  hand, 
if  grating  OB  is  read  out  in  the  same  direction,  the 
magnitude  of  the  reading  vector  should  be  changed  to 
OD  (DF  is  the  perpendicular  bisector  of  OB),  and  the 
diffracted  wave  vector  deviates  from  AC  by  2A0. 
Thus,  by  changing  the  wavelength,  we  can  read  differ¬ 
ent  gratings  in  the  same  reading  direction.  S’ ice 
different  gratings  cause  different  diffraction  angles, 
the  reconfigurable  optical  interconnections  can  be  re¬ 
alized  in  this  simple  fashion.  The  interconnection 
pattern  is  previously  recorded  in  the  volume  hologram, 
for  which  the  angular  and  wavelength  selectivities 
should  be  high  enough  to  prevent  crosstalk  between 
channels.  We  assume  that  the  range  of  tunable  wave¬ 
lengths 

‘ax' 

_  x  _ 

of  a  laser  diode  is  '-0.01  (Ref.  5)  (i.e.,  AX/X  ~  0.01),  and 
the  required  angular  deviation  is  2Aa  =  0.2  rad,  then  a 
should  be  >84°  for  transmission  holograms  [from  Eq. 
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(2)].  We  note  that  this  angle  requires  a  large  size 
recording  crystal  (i.e.,  about  a/cosa,  where  a  is  the  laser 
beam  diameter) .  However,  for  reflection  holograms,  a 
is  ~45°  [from  Fig.  2(b)],  which  is  in  fact  the  optimum 
angle  for  angular  selectivity,  and  this  angle  is  also  more 
convenient  to  work  with  in  practice. 

The  choice  of  thickness  d  of  the  photorefractive 
crystal  should  allow  the  bandwidth  of  the  hologram  to 
be  equal  to  or  greater  than  the  signal  bandwidth.  If 
the  modulation  frequency  of  the  laser  diode  is  A  f. 

{?}-¥■ 

where  c  is  the  velocity  of  light  and  subscript  s  signifies 
the  signal.  By  substituting  this  relationship  in  Eq. 
(16),  we  see  that  the  thickness  d  of  the  crystal  should 
satisfy  the  following  inequality: 


Notice  that  the  corresponding  angular  selectivity  of 
(A a)r  can  be  obtained  from  Eq.  (15).  By  taking  (A a)c 
=  3(Aa)r  as  the  angular  separation  between  chan-  ols, 
the  crosstalk  due  to  the  sidelobss  of  the  sine  function 
would  be  less  than  —20  dB,  where  the  subscript  c 
signifies  the  channel.  The  wavelength  interval 


can  be  derived  from  Eq.  (6),  and  it  can  be  used  as  the 
longitudinal  mode  spacing  of  laser  diodes  for  wave¬ 
length  tuning.  This  would  require  the  effective  opti¬ 
cal  cavity  length  to  be 


n,L  - 


/k\ ' 

‘UJc 


(24) 


where  ni  is  the  refractive  index  of  the  laser  medium, 
and  1.  is  the  physical  length  of  the  cavity. 

As  an  example  using  LiNbOa  crystal  (no  =  2.28)  as 
the  recording  medium,  a  ■  •  45°,  A/  =  10  GHz,  and  X  = 
680  mr.,  the  relative  signal  bandwidth  is 

[^.2.4*10-*. 


and  the  thickness  of  the  photorefractive  crystal  dmai  = 
13.8  mm,  the  angular  selectivity  is  (Aa)r  =  2.1  X  10-4, 
and  the  angular  separation  between  channels  is  (Aa)c 
=  6.3  X 10~4.  For  0.1-raa  angular  deviation,  one  would 
have  about  160  channels  with  a  wavelength  tuning 
range  AX/X  of  ~0.01.  The  wavelength  interval  be¬ 
tween  channels  can  be  shown  as 


and  the  effective  cavity  optical  length  of  the  laser  diode 
is  tiiL  =  5  mm.  If 


is  used  as  the  required  relative  wavelength  stability,  we 
would  have  the  required  wavelength  stability  of  laser 


Fig.  7.  Reconfigurabl,  optica}  crossban  C,  c-cut  LiNbO.i  crystal; 
L\,Li,  collimating  and  f  musing  lens;  P\ ,  Pi,  input  and  output  planes; 
LA,  DA,  laser  diode  ai  ay  and  line  detector  array  on  the  planes  Pi 
tad  Pi. 


diodes,  such  as  (AX)stabiiity  =  0.16  nm.  This  is  possible 
for  a  single  laser  diode  at  the  present  time. 

A  possible  configuration  of  a  reconfigurable  optical 
crossbar  structure  is  shown  in  Fig.  7.  Multiple  reflec¬ 
tion  gratings  are  written  on  a  piece  of  z-cut  LiNbO.-} 
crystal.  The  writing  light  can  be  derived  from  a  dye 
laser  or  a  He-Ne  laser  (X  —  632.8  nm),  and  the  reading 
light  can  be  derived  from  an  array  of  laser  diodes  (X  = 
680.0  nm).  The  wavelength  difference  between  write- 
in  and  reading  may  be  compensated  by  modifying  the 
write-in  angle  according  to  the  dispersion  relation. 
The  writing  process  is  performed  at  a  fixed  writing 
angle,  and  multiple  exposures  are  carried  out  by  rotat¬ 
ing  the  crystal  with  an  angular  separation  equal  to  the 
required  channel  separation.  The  exposure  procedure 
given  by  Strasser  et  alJ  should  be  adopted  to  make  the 
diffraction  efficiencies  of  different  channels  uniform. 
After  writing,  a  fixing  procsvwe  as  described  by  Stae- 
ble  et  al8  should  be  used  to  ■:  the  holograms  in  the 
crystal.  A  linear  laser  diode  array  is  arranged  along 
the  nondispersion  direction  at  the  input  plane.  Since 
it  is  in  the  nondispersion  direction,  there  would  be  no 
wavelength  tunable  range  limitation  for  the  separa¬ 
tions  between  laser  diodes,  provided  the  total  extent  of 
the  array  is  within  the  paraxial  approximation.  A 1-D 
line  detector  array  is  placed  at  the  output  plane  with  a 
separation  of  the  line  detectors  equal  to  2  times  the 
channel  separation,  as  previously  given.  Different  la¬ 
ser  diodes  can  use  the  same  sel  ,>f  ratings  in  the 
crystal.  In  the  nondispersion  direction,  the  light 
beams  are  reflected  according  to  the  law  of  reflection. 
By  independently  controlling  the  wavelength  of  the 
laser  diodes,  a  reconfigurable  electrooptical  crossbar 
architecture  can  be  realized  in  practice. 

IV.  Reconfigurability  with  Spatial  Division 
Another  possible  architecture  for  reconfigurable  in¬ 
terconnections  with  volume  holograms  is  spatial  divi¬ 
sion  instead  of  wavelength  tuning.  A  page  oriented 
holographic  setup  is  shown  in  Fig.  8.  We  shall  apply 
the  pinhole  hologram  technique  as  proposed  by  Xu  et 
al?  into  a  photorefractive  crystal.  The  angular  selec¬ 
tivity  of  the  recorded  volume  hologram  would  have  a 
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larger  storage  capacity  compared  with  thin  holograph¬ 
ic  plates.  The  recording  arrangerr  it  is  shown  in  Fig. 
8(a),  in  which  an  SLM  is  in  the  focal  plane  of  condenser 
lens  L\  and  the  interconnection  pattern  masks  are 
placed  at  page  plane  P.  Object  beam  B  is  focused  by 
Lu  after  passing  the  SLM  the  beam  is  directed  toward 
the  recording  medium.  We  note  that  the  SLM  is  used 
to  generate  a  movable  pinhole  that  allows  only  the 
object  beam  to  pass  in  one  direction.  In  other  words, 
the  interconnection  mask  is  illuminated  by  the  object 
beam  in  one  direction,  where  the  pinhole  of  the  SLM  is 
set  at  a  spatial  location  to  allow  the  object  beam  to  pass 
through.  It  can  be  seen  that  a  set  of  interconnection 
masks  can  be  encoded  in  the  crystal  for  a  given  refer¬ 
ence  beam  A  by  moving  the  SLM  pinhole  position  and 
changing  the  object  beam  B'.  Similarly,  another  set  of 
interconnection  pattern  masks  can  be  encoded  for  an¬ 
other  reference  beam  A'  and  so  on.  In  the  readout 
process,  a  1-D  laser  diode  array  is  placed  at  the  front 
focal  plane  Q  of  the  collimating  lens  L2  as  shown  in  Fig. 
8(b).  Each  diode  generates  a  reading  beam  that  is 
conjugate  to  a  specific  reference  beam  A.  When  the 
SLM  pinhole  is  set  at  one  position,  a  set  of  interconnec¬ 
tion  patterns  will  be  diffracted  and  projected  onto  page 
plane  P.  As  the  pinhole  position  is  moved,  the  inte- 
connection  patterns  would  be  changed.  Thus  we  see 
that  the  interconnections  between  the  laser  diode  ar¬ 
ray  and  the  page  plane  can  be  made  reconfigurable  by  a 
programming  SLM. 

Unlike  the  wavelength  tuning  reconfigurable  inter¬ 
connection,  the  spatial  division  reconfiguration  re¬ 
quires  low  wavelength  selectivity  (not  sensitive  to  the 
wavelength).  Based  on  the  comparisons  of  volume 
holograms  discussed  in  Sec.  II,  we  should  use  transmis¬ 
sion  type  holograms  for  interconnections.  To  have  a 
higher  angular  selectivity,  the  average  write-in  angle 
(2a)  should  be  ~90°.  If  the  thickness  of  the  photore- 
fractive  crystal  (i.e.,  LiNbOs)  is  ~l-2  cm,  the  band¬ 
width  of  the  hologram  would  be  much  wider  than  the 
signal  bandwidth  as  can  be  seen  in  Fig.  4(a).  With 
reference  to  Fig.  3(a),  the  angular  selectivity  is  found 
to  be  ( Aa)t  =  5 X/d  -  3.4  X  10~4.  If  3(Aa)t  is  used  as  the 
channel  separation  and  assuming  a  30°  angular  range, 
there  would  be  ~103  useful  interconnection  channels 
in  the  photorefractive  crystal.  Due  to  the  degenera¬ 
tion  of  Bragg  diffraction,  only  a  1-D  laser  diode  can  be 
used.  If  the  full  range  of  NX  N  SLM  pinholes  is  used , 
the  total  number  of  interconnection  patterns  would  be 
M  -  N2  X  K,  where  K  is  the  number  of  channels  (i.e., 
the  number  of  laser  diodes). 

In  principle,  this  architecture  can  be  used  for  mas¬ 
sive  information  storage.  As  an  example  if  N  =  32,  K 
-  103,  the  total  number  M  of  patterns  (or  pictures) 
would  be  106,  which  is  about  the  capacity  of  a  10-h  TV 
program.  This  architecture  does  not  require  a  high 
space-bandwidth  product  of  the  SLM  (only  32  X  32). 
The  requirement  of  the  pinhole  size  is  about  mXF/a, 
where  mXmis  the  number  of  pixels,  a  is  the  size  of  the 
pictures,  F  is  the  focal  length  of  the  condenser  lens  L\. 
The  size  of  the  SLM  is  about  mNXFla.  For  example,  if 
m  =  512,  N  -  32,  X  =  680  nm,  Fla  =  2,  the  required  size 


Fig.  8.  Geometry  for  reconfigurable  interconnections  with  spatial 
division:  (a)  recording  setup;  (b)  reading  setup.  L\,  condensing 
lens;  L?,  collimating  lens:  SLM,  spatial  light  modulator:  C.  recording 
media;  P,  page  plane;  Q.  laser  diode  plane. 


of  the  SLM  is  ~23  X  23  mm.  The  main  advantage  of 
this  architecture  is  its  rapid  and  random  access  to  the 
pictures  in  contrast  with  the  sequential  access  in  opti¬ 
cal  disks. 

In  practice,  the  total  number  M  of  interconnection 
patterns  which  can  be  recorded  in  the  volume  holo¬ 
gram  is  basically  limited  by  the  maximum  index  modu¬ 
lation  ri\  of  the  photorefractive  material.  With  refer¬ 
ence  to  Ref.  10,  the  number  of  interconnection 
patterns  is  about 


For  example,  given  An  ~  10-3,  if  v  ~  10-3,  d-  2  cm, 
and  X  *  680  nm,  for  an  ion-doped  LiNbO.-)  photorefrac¬ 
tive  crystal,  the  total  number  of  interconnection  pat¬ 
terns  would  be  M  ~  1.2  X  103.  Recently,  photorefrac¬ 
tive  fibers  have  been  successfully  grown.  The  fibers 
can  be  used  to  synthesize  a  large  volume  of  photore¬ 
fractive  material,11-12  which  is  not  possible  to  grow  in  a 
single  crystal  form.  The  effective  thickness  d  of  the 
material  in  Eq.  (25':  can  be  expected  to  increase  by  '.-2 
orders  of  magnitude.  In  addition,  r.ew  murei.  with 
higher  ni  can  also  be  expected  (*-.,*..  organic  matori 
als).13  Therefore,  the  material  limitation  of  2q.  *25' 
may  be  alleviated  in  the  future.  On  the  other  hand, 
we  would  like  to  point  out  that  the  crosstalk  from 
third-order  diffraction,  scattering,  internal  reflections, 
and  noise  behavior  of  the  volume  holograms  should 
be  investigated  before  practical  application  of  angular 
or  wavelength  multiplexing  in  photorefractive  materi¬ 
als. 

V.  Experimental  Demonstrations 

The  experimental  setup  for  reconfigurable  intercon¬ 
nections  with  wavelength  tuning  is  shown  in  Fig.  9. 
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Fig.  9.  Experimental  setup:  C,  y-cut  LiNbOj:  PBS.  polarizing 
team  splitter,  A/2,  halfwave  plate:  Af,  mirror:  L,  focusing  lens;  P. 
output  plane. 


Fig.  10.  Readout  light  spots  with  eight  different  wavelengths. 


The  laser  beam  is  divided  into  two  parts  by  a  polarizing 
beam  splitter.  A  halfwave  plate  was  used  to  rotate  the 
polarization  of  one  beam  so  that  the  reflection  holo¬ 
gram  was  formed  in  ordinary  polarization.  A  piece  of 
y-cut  J.iNhOr,  crystal  1  mm  thick  was  employed  as  the 
recording  medium.  The  write-in  angle  2a  of  beams  A 
ind  B  is  50°.  Since  we  do  not  have  a  z-cut  crystal, 
t<~  nave  the  grating  vector  component  on  the  c-axis,  the 
writing  arrangement  should  be  asymmetrical.  Beam 
A  is  arranged  to  be  perpendicular  co  t'.ip  crystal  sur¬ 
face.  The  wavelength  rsed  for  writing  is  632.8  nm. 
Eight  exp  osure  were  made  by  -otating  the  crystal  with 
a  0.8*  step.  A  tunable  dye  laser  in  used  to  mad.  the 
hologram  in  the  smae  direction  .<s  benn  B.  The  recon¬ 
structed  light  field  is  collected  by  focur,;;  g  lens  L  onto 
output  plane  P.  It  is,  ti=.re:>’-e.  apparert  that  holo¬ 
graphic  gratinrs  -an  be  i  ’ad  out  one  “tu  time  by  simply 
tuning  the  wavelength  of  the  dve  laser.  The  readout 
spots  from  eight  grating;  encoded  in  the  crystal  using 
different  wavelengths  are  shown  in  Fig.  ID.  Figure  1 1 
shows  a  linear  relationship  c  f  rb0  reconstruction  wave¬ 
length  vs  the  deflection  angle,  it  »s  clear  that  the 
write-in  angle  (i.e.,2a),ca!ei'kt-d  from  the  slope  of  the 
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Fig.  U.  Reconstruction  wavelength  as  a  function  of  the  deflected 
sngle. 


(a) 


(b) 

Fig.  12.  Reconftgurable  interc  mnections  with  spatial  division:  (a) 
reconstructed  patterns  on  the  p.’ge  plane  by  three  different  reading 
directions  with  the  same  pinhole  position;  (b'  reconstructed  six  focal 
spots  on  the  SLM  plane  by  one  reading  beam. 


straight  line  using  the  dispersion  relation  of  Eq.  (2),  is 
given  by  2a  =  148.5°. 

The  experimental  setup  for  spatial  division  reconfi¬ 
gurable  interconnections  is  basically  the  same*  as 
shown  in  Fig.  8,  except  that  one  laser  beam  illumina¬ 
tion  in  different  directions  is  used  instead  of  a  laser 
diode  array.  In  our  experiment  we  employed  a  mov¬ 
able  pinhole  as  the  SLM  pinhole.  The  moving  step  of 
the  pinhole  is  ~l  mm:  the  focal  length  of  the  condenser 
lenc  is  250  mm.  Figure  12(a)  shows  three  reconstruct¬ 
ed  patterns  for  the  same  pinhole  position  in  three 
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different  reading  directions.  The  angular  separation 
of  the  reading  beams  is  ~1°.  Figure  12(b)  shows  rhe 
reconstructed  six  f-cal  spots  at  the  SLM  plane  with  a 
single  reading  direction.  If  these  s*'x  focal  spots  are 
projected  onto  the  page  plane,  six  interconnection  pat¬ 
terns  would  overlap.  To  separate  the  overlapping  pat¬ 
terns,  a  pinhole  can  be  used  to  select  one  focal  spot  at  a 
time.  Thus  nonoverlapping  patterns  can  be  sequen¬ 
tially  displayed  on  the  page  plane. 


VI.  Conclusion 

We  have  analyzed  the  dispersion,  angular,  and  wave¬ 
length  selectivities  for  transmission  and  reflection  vol¬ 
ume  holograms.  The  refraction  on  the  surfaces  of  the 
crystal  plays  an  important  role  in  the  performance  of 
the  holograms.  Since  the  reflection  holograms  have 
higher  and  more  uniform  dispersions,  it  is  more  suit¬ 
able  for  the  purpose  of  reconfigurable  optical  intercon¬ 
nections  with  a  limited  tunable  wavelength  range.  It 
is  possible  to  satisfy  the  communication  channel  band¬ 
width  if  the  thickness  of  the  crystal  is  properly  select¬ 
ed.  In  principle,  a  large  number  of  reconfigurable 
optical  interconnections  with  high  channel  bandwidth 
(e.g.,  1010  Hz)  can  be  achieved  bv  using  reflection  holo¬ 
grams. 

We  have  also  shown  that  the  transmission  volume 
hologram  is  more  suitable  for  the  spatial  division 
scheme.  Since  the  transmission  volume  hologram  is 
encoded  in  the  form  of  pinhole  holograms,  the  optical 
interconnection  pattern  from  each  laser  diode  illumi¬ 
nation  can  easily  be  reconfigured  by  a  programmable 
SLM.  Nevertheless,  for  both  the  wavelength  tuning 
and  spatial  division  schemes,  the  maximum  number  of 
interconnections  stored  in  a  volume  hologram  is  limit¬ 
ed  by  the  maximum  index  modulation  of  the  photore- 
fractive  material. 
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Abstract— A  potm  diffraction  iBttrfsromrttr  is  used  to  examine  (he 
phase  nominifomnrt  of  a  liquid  ciTstal  tdrriskm  ( LCTV).  The  resulting 
interference  pattern  :s  then  binarized  and  written  back  onto  the  LCTV 
for  phase  compensation.  A  binary  phase  joint  transform  correlation 
experiment  is  performed  using  the  binary  phase  corrected  LCTV  to 
demonstrate  the  applicability  of  this  technique. 


Introduction 

COMMERCIALLY  available  liquid  crystal  televisions 
(LCTV')  have  been  used  as  an  inexpensive  but  effective 
spatial  light  modulators  (SLM)  for  many  optical  signal  pro¬ 
cessing  applications  {l]-(3].  This  device  can  also  be  operated 
as  a  binary  phase  only  SLM.  by  orientating  the  second  ana¬ 
lyzer  to  be  perpendicular  to  the  bisector  of  the  two  transmitted 
polarization  states  of  the  LCTV.  as  illustrated  in  Fig.  1  [4J. 
However,  the  nonuniform  thickness  of  the  liquid  crystal  dis¬ 
play  panel  causes  phase  distortion  which  prohibits  the  direct 
application  of  the  LCTV’s  in  certain  optical  processing  ex¬ 
periments  that  require  high  phase  precision.  Many  researchers 
have  immersed  the  display  panel  in  a  liquid  gate  tilled  with  in¬ 
dex  matching  oil.  thus  removing  die  external  phase  distortion. 
Casasent  et  al.  (5]  ano  Yu  et  al.  [6]  have  also  proposed  the  use 
of  a  phase  conjugation,  method  for  alleviating  t  is  problem. 

An  article  by  Kim  et  al.  (7J  suggested  correcting  Lhe  phase 
distortion  of  an  SLM  by  writing  the  binari  *d  phase  distortion 
function  back  onto  the  device.  In  this  let.  ^r.  a  closed-bop 
binary  phase  correction  technique  is  demons. 'ated  which  uses 
a  point  diffraction  interferometric  technique  ;8|  instead  of  a 
Mach-Zehndcr  interferometer  as  mentioned  in  [7],  A  closed- 
loop  and  on-line  architecture  is  introduced  so  the  optimum 
correction  function  can  be  generated. 

Point  Diffraction  Interferometer 

The  point  diffraction  interferometer  was  first  proposed  by 
Smart  and  Strong  [8]  as  a  technique  for  measuring  the  phase 
variation  across  a  wavefront,  the  principle  of  wh*ch  ;s  illus¬ 
trated  in  Fig.  2.  The  wavefront  to  be  examined  brought 
to  a  focal  spot  by  a  converging  lens,  and  i»  then  transmute.. 
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work  -*as  supported  by  the  U.S.  Army  Missile  Command  through  U.S.  Art.  v 
Research  Office  under  Contract  DAAL03-S7  K-0S47 
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Fig.  2.  Principle  operation  of  a  point  diffracaor.  .Merferomeier. 


with  reduced  amplitude  through  an  absorbing  film  placed  at 
the  focal  plane.  The  film  has  a  small  transparent  pinhole  act¬ 
ing  as  a  point  source  which  produces  a  spherical  reference 
wavefront.  The  wavefront  under  examination  then  interferes 
with  the  reference  wave  when  being  recorded  by  a  detector 
placed  behind  the  absorbing  film.  The  usual  fueus  shift  and 
tilt  of  the  reference  wavefront  are  made  possible  by  displacing 
the  pinhole  longitudinally  and  laterally,  respe-rtivciy.  from  the 
focal  spot  of  the  optical  system.  However.  >n  the  application 
discussed  here,  it  is  desirable  to  place  the  pinhole  at  the  center 
of  the  focal  plane  since  a  tilt  or  focal  shift  would  introduce 
additional  interference  fringes  that  cannot  be  compic'ely  com¬ 
pensated  for  by  a  binary  phase  correction  function-  Since  the 
point  diffraction  interferometer  is  a  common-path  interferom¬ 
eter  the  fringes  are  s'ablc  against  vibratic-n.  and  wtde-band 
ligh:  sources  can  be  used.  The  simplicity  of  the  point  diffrac¬ 
tion  interferometer  architecture  is  aiso  one  of  s  major  merits. 

Binary  Phase  Correction 

Let  i  unity  amplitude  function  htx.  }  )  represent  fnc  nonum- 
form  phase  distribution  on  the  LCTV’s  display  paid.  If  ’he 
complex  conjugate  of  this  function.  h’(x,  y).  is  displayed  on 
ti.c  LCTV.  tiiei!  uie  uvci.ii!  tiansitiiiuiicc  futtctu n  would  i.-c 
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Fig.  3.  (a)  Computer-simulated  result  for  binary  phase  correction  of  a  phase 
distorted  surface.  Each  surface  is  made  from  a  circular  surface  i  radius  =  40 
pixels)  enclosed  in  a  256  x  256  image  array.  A  set  of  Gaussian  distributed 
phase  angle  values  t  standard  deviation  =  a  in  r  tad.  unit)  arc  first  gener¬ 
ated  as  seeds.  Toe  seed  angles  are  then  assigned  to  every  5  *  5  array  on 
the  circular  surface.  Finally,  a  neighborhood  averaging  filter  is  applied  to 
generate  the  distorted  surtace.  Degree  of  phase  distortion  as  labeled  on  the 
jr-axis  equals  to  0.1  a.  The  y-axis  facets  the  normalized  peak  intensity  of 
the  plane  wave  response.  When  the  degree  of  phase  distortion  is  below  5 
units,  the  phase  distortion  on  the  surface  basically  has  a  standard  deviation 
less  than  x/2:  therefore,  the  binary  phase  cornea  ion  scheme  has  little  ef¬ 
fect  in  restoring  the  surface,  (b)  Deviation  of  die  peak  from  the  origin  as  a 
function  of  the  degree  of  phase  distortion.  Angie  of  deviation  of  the  peak 
can  be  related  to  the  deviation  distance  .V  tin  pixel  unnsi  by  tire  equation 
o$  =  (iV/2S6)(X/our),  where  ox  is  the  width  of  a  pixel. 
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Fig.  4.  Experiments!  setup  for  a  closed-loop  binary  phase  correction  system 
using  a  point  diffraction  interferometer.  The  polarizer  a  oriented  at  die 
proper  angle  to  achieve  binary  phase  oniy  modulation. 


To  investigate  the  effectiveness  of  this  binary  phase  correc¬ 
tion  technique,  a  computer  simulation  was  carried  out.  A  set  of 
randomly  phase-distorted  surfaces  characterized  by  different 
levels  of  phase  distortion  were  first  generated,  and  their  plane 
wave  responses  were  computed  by  using  an  FFT  routine.  The 
intensity  and  position  of  the  focal  peak  at  the  Fourier  plane 
were  recorded  as  a  function  of  the  degree  of  phase  distortion. 
The  lower  curve  in  Fig.  3(a)  shows  that  the  peak  intensity 
decreases  rapidly  as  the  degree  of  phase  distortion  increases. 
Likewise,  the  location  of  the  peaks  also  shift  randomly  as 
shown  in  Fig.  3(b).  The  binary  phase  correction  scheme  is 
then  employed,  and  the  restored  peak  intensity  is  plotted  in 
the  upper  curve  of  Fig.  3(a).  It  shows  that  the  restored  peak 
intensity  is  improved  up  to  65%  of  its  original  value  even  when 
the  input  function  is  under  severe  phase  corruption.  It  can  aiso 
be  seen  in  Fig.  3(b)  that  the  binary  phase  correction  scheme 
successfully  restores  the  peak  back  to  its  proper  position. 


Experimental  Demonstrations 

Fig.  4  shows  the  experimental  setup  for  implementing  this 
proposed  technique.  A  zoom  iens  is  placed  behind  the  absorb¬ 
ing  film  so  that  the  LCTV  panel  can  be  mapped  accurately 
onto  the  C CD  camera  Cl .  When  the  LCTV  is  turned  off.  the 
interference  pattern  recorded  by  Cl  is  given  by 


h(x,  y)h'(x,  y)  =  I,  i.e.,  phase  distortion  of  the  display 
panel  would  be  removed.  However,  the  LCTV.  operating  in 
a  phase  modulation  mode,  can  only  generate  two  phase  val¬ 
ues  (0  and  -  phase).  Therefore,  the  proper  phase  correction 
function  to  be  used  is  the  binarized  function  given  by 

{-ril  if  Re  fh'fx,  >’)}  >  0: 

(I) 

- 1  otherwise. 

Conceptually,  this  is  equivalent  to  adding  a  zero  phase  to  the 
region  of  the  SLM  if  the  phase  distortion  is  in  the  first  and 
fourth  quadrant,  and  a  -  phase  to  the  region  if  the  phase  is  in 
the  second  and  third  quadrant.  Although  perfect  phase  com¬ 
pensation  cannot  be  achieved,  phase  distortion  of  she  display 
panel  after  correction  is  confined  to  the  region  from  --/2  to 
-w/2  such  that  all  pixel  elements  of  the  LCTV  make  construc¬ 
tive  contributions  to  the  output. 


>’)  =  \h(x,  y)-FC|*=C.  -fC>  Re  {/r(.r,  >•)}  2) 

where  C  represents  the  amplitude  of  the  reference  wave  dut  to 
the  pinhole  source  on  the  absorbing  film.  Since  Re  {hix.  u} 
and  Re  {h*(xr.  yt)  are  equivalent,  the  binary  phase  correc¬ 
tion  function  hUx.  y)  can  be  obtained  from  I(x,  y)  with  a 
proper  threshold.  To  observe  the  phase  correction  result  when 
/ m(x,  >•)  is  displayed  on  the  LCTV.  a  beamsplitter  is  intro¬ 
duced  so  ihat  the  focal  spot  peak  intensity  can  be  recorded  on 
CCD  camera  C2.  Thus,  the  performance  of  the  phase  correc¬ 
tion  function  can  be  evaluated  as  a  function  of  the  focal  spot 
intensity.  By  adjusting  the  threshold  level  of  the  binarization 
process  using  this  on-iine  and  closed-loop  architecture,  the 
optimum  binary  phase  correction  function  can  be  obtained. 

i  p  f,  ( V  iri  —  H*/v»  •*} 

In  the  experiment,  an  Epson  color  LCTV  model  ET708  was 
used.  The  dimensions  of  the  display  panel  arc  50  mm  x  65 
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F:e.  5.  i  ai  Correction  function  emptovrvi  for  the  LCTV  under  iejt.  i  b!  Plane 
ware  rewsocie  of  the  phase  vlin.wtcd  LCTV  before  tsajry  phase  correct  ~ys. 
ana  tel  after  correciiosi. 

mm.  the  resolution  is  238  pixels  a  -442  pixeis.  The  modu¬ 
lation  angle  between  the  two  polarization  states,  given  as  0 
in  Fig.  1 .  was  measured  to  be  about  46  in  this  experiment  set- 
tine.  The  aperture  used  in  the  point  diffraction  interferometer 
was  made  by  burning  a  pinhole  (~5  ami  on  a  =2Yv  transmit¬ 
tance  film.  Fig.  5(a)  shows  the  optimum  binary  phase  correc¬ 
tion  function  thus  generated.  The  focal  spots  recorded  by  C2 
before  and  after  phase  correct! jn  arc  shown  in  Figs.  5(b)  and 
ic>.  respectively.  It  can  be  seen  that  the  phase  compensation 
produces  a  sharper  plane  wave  response  whose  mtcnsirv  is  al¬ 
most  one-third  higher  than  'hat  without  phase  compensation. 

A  binary  image  fix.  vy  can  also  be  written  onto  the  binary- 
phase  corrected  LCTV  for  image  processing.  In  this  case, 
the  final  phase  image  displayed  on  the  LCTV  is  given  by 
fix ,  y)hL(X.  y).  Since  both  oi  these  functions  arc  binary  m 
nature,  the  multiplicative  operation  can  be  repiac; .  by  an  Ex¬ 
clusive  or  operation  which  facilitates  digital  !.".■<>'  tv  ration. 
To  demonstrate  that  the  binary  phase  corrccte.  •  C,  v  can  be 
used  in  binary-input  image  processing  appiuraiirn*.  a  bmary 
phase  only  joint  transform  correlation  |9|  expenn-.— f  per¬ 

formed.  An  LCTV  (immersed  in  a  liquid  gate  „sed  at 
the  input  plane  to  display  two  identical  nonbir.  y  .,iT)p:.tudc 
modulated  input  imar  *.  and  a  CCD  camera  p'  ;cd  at 
she  Fourier  plane  to  vord  the  joint  transform  'v  spec 
trum.  The  power  sp  urum  thus  obtained  was  ,’-en  narize-d 
by  using  a  microcomputer  and  displayed  on  .  ary  phase 
corrected  LCTV  ).■•*  ne  second  Fourier  transformation.  The 


(c)  (d) 


Fig.  6.  (ai  A  binarized  joint  transform  purer  spectrum  obtained  a:  the 
rouner  plane of  2 1TC  whose  :r.pu:  is  rwo  identical  maces.  ih>  The  Exclu¬ 
sive  o*  resul:  berween  us*  power  iwom  in  u>  and  the  boary  coereenoo 
function  in  Fie.  5ui.  icl  Correaiicr.  esipei  when  (ai  is  displayed  or,  she 
LCTV.  The  -dc  is  ai  the  center  of  sis  phosoeraph.  a  rule  she  two  peaks  or, 
the  sides  are  she  cerreiauen  fenctsam.  id»  Correlasicn  outset  when  ibi  is 
displayed  on  use  LCTV 

binarized  joint  transform  power  spectrum  is  shown  in  Fig. 
6ta).  while  Fig.  6<b)  shows  the  Exclusive  or  result  between 
the  spectrum  and  the  phase  correction  function.  Fig.  6(c)  and 
(d)  shows  the  output  correlation  results  before  and  after  the 
binary  phase  correction  function  is  applied-  Resuits  are  similar 
to  those  shown  in  Fig.  5:  sharper  correlation  peaks  whose  in¬ 
tensities  arc  about  one-third  higher  than  those  without  binary 
phase  correction  are  observed. 

The  same  phase  correction  principle  can  be  applied  to  cor¬ 
rect  the  phase  distortion  of  an  optical  signal  processing  system 
instead  of  the  LCTV  itscif.  The  TV  panel  is  used  as  a  real¬ 
time  addressable  binary  phase  filter  solely  for  phase  correction 
purposes.  To  illustrate  this  idea,  the  LCTV  display  wras  im¬ 
mersed  into  a  home-made  liquid  gate.  The  nonumform  phase 
distortion  of  the  LCTV  was  corrected  by  the  index  matching 
oil.  However,  the  undesirable  curvature  of  the  liquid  gate's 
optical  window  introduced  an  astigmatic  aberration,  whose 
interference  pattern,  obtained  using  the  point  diffraction  inter¬ 
ferometer.  is  shown  in  Fig.  “fai.  The  focal  spot  structures, 
recorded  at  the  proper  focal  plane  before  and  after  binary 
phase  correction  by  the  LCTV.  are  shown  in  Fig.  7(b)  and  tc). 
respectively.  The  programmable  phase  filter,  s.e..  the  LCTV. 
effectively  removed  the  aberration,  and  a  sharp  focal  spot  was 
observed  at  ihe  correct  focal  plane. 

Conc'i.l  sum 

Binary  phase  correction  oi  an  LCTV  using  a  point  diffrac¬ 
tion  interferometer  has  been  demonstrated.  The  use  of  a  point 
diffraction  interferometer  offers  the  simplicity  ai.J  advantages 
of  a  common-path  inierleromcter  tor  examining  the  phase 
nonumfotmitv  of  the  LCTV.  while  the  on-line  and  closed- 
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fig.  7.  (a)  Binary  eorrectkm  fuactwr,  for  an  astigmatic  aberrated  cpekal 
system,  tb)  Pane  »ave  response  of  she  aberrated  system  before  bicaiv 
piiase  correction,  and  tc)  after  corrccdoc. 


loop  architecture  is  capable  of  generating  the  optimum  phase 
correction  function.  Both  computer  simulations  and  exper¬ 
imental  results  demonstrate  the  effectiveness  of  the  binary 
phase  correction  scheme.  The  results  obtained  from  the  bipo¬ 
lar  phase  only  joint  transform  correlation  reveal  the  feasibility 
of  this  phase  correction  technique  in  actual  applications.  Men¬ 
tion  must  be  made  that  this  binary  phase  correction  scheme 
can  be  used  to  compensate  for  the  phase  distortion  of  a  gen¬ 
eral  optical  signal  processing  system.  Other  phase  modulat¬ 
ing  real-time  addressable  SLM”s  which  have  a  higher  contrast 
ratio  and  a  larger  space-bandwidth  product,  such  as  the  mas- 
netooptic  SLM's  ;  10)  or  liquid  crystal  light  valves  111),  may¬ 
be  employed  to  produce  even  better  results. 
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ABSTRACT 

This  article  will  discuss  a  two-category  image  classification  which 
has  been  performed  on  a  joint  transform  correlator  UTC)  by  using 
the  Kittler-Young  transform.  The  property  of  optimally  utilizing  the 
class  means  and  variances  in  the  Kittler-Young  transform  has  en¬ 
abled  a  high  space-bandwidth-product  operation.  The  digitizing  ef¬ 
fect  of  the  liquid  crystal  television  has  been  utilized  to  increase  the 
correlation  peaks. 

The  application  of  statistical  pattern  recognition  introduces 
effective  feature  extraction  to  optical  systems  (1).  In  statistical 
pattern  recognition,  principal  component  analysis  (2)  is  widely 
used  (e.g.,  Clafic  [3],  Selfic  (2],  the  Fukunaga-Koontz  (FK) 
transform  [4],  and  the  Kittler-Young  (KY)  transform  [5]). 
The  FK  transform  has  been  implemented  by  Leger  and  Lee 
for  64  x  64  images  on  the  Vander  Lugt  correlator  (6j.  The 
first  three  algorithms  mentioned  above  have  a  common  char¬ 
acteristic.  The  feature  selection  criteria  are  based  on  the 
corresponding  eigenvalues  generated  from  the  principal 
component  analyses.  However,  the  KY  transform  goes  one 
step  further  in  using  the  eigenvalues,  such  that  a  larger  space- 
bandwidth-product  (SBP)  process  can  be  performed  in  pat¬ 
tern  recognition.  This  article  presents  the  implementation  of 
,*the  KY  transform  on  the  joint  transform  correlator  (JTC)  [7. 
8]  to  perform  a  two-categorv  image  classification.  The  JTC 
implementation  of  the  KY  transform  can  eliminate  the  pre- 
process  of  matched  filter  synthesis,  which  is  requited  in  the 
Vander  Lugt  correlator.  Furthermore,  the  resolution  require¬ 
ment  at  the  frequency  plane  in  JTC  Is  much  lower  than  the 
Vander  Lugt  correlators,  such  that  the  direct  application  of 
spatial  light  modulators  (SLM)  at  the  frequency  plane  is  pos¬ 
sible.  As  a  result,  high  SBP  image  classification  is  achieved. 
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The  KY  transform  is  a  linear  transformation  to  extract 
features  from  images.  The  transform  matnx  for  the  KY  trans¬ 
form  is  constructed  in  a  training  process.  Training  images  of 
n  x  n  pixels  used  in  the  training  process  are  lexicographically 
scanned  and  stored  as  column  vectors  xU]j  of  dimension  N 

(=n:),  where  indices  i  (  =  1.  2 . K)  and  j  (1.  2 . 

M)  are  the  class  and  image  indices,  respectively.  The  first 
step  in  constructing  the  KY  transform  matrix  is  to  diagonalize 
the  covariance  matrix  C  (5)  of  the  training  images,  in  which 
the  eigenvector  matrix  and  the  eigenvalue  matrix  A  of  the 
covariance  matrix  C  are  obtained.  Each  eigenvalue  /.» is  thev 
average  class  variance  of  the  corresponding  feature.  In  order 
to  normalize  the  class  variances  of  the  features,  a  whitening 
transform  is  performed  [9]  in  the  second  step.  After  two  suc¬ 
cessive  linear  transforms  with  transform  matrices  4>  and 
A'1-,  image  vectors  x  are  mapped  onto  feature  vectors  v.  The 
covariance  matrix  in  y  space  becomes  the  identity  matrix  with 
all  the  average  class  variances  normalized  to  unity.  In  the  final 
step,  the  correlation  matrix  /?,  [5]  in  y  space  is  diagonalized, 
and  the  overall  KY  transform  matrix  T  is  thus  formed: 

T  =  4>A''  2B  =  (t„  t3 . t,M).  (1) 

where  B  is  die  eigenvector  matrix  of  the  correlation  matrix 
R..  and  t,  is  the  .<th  column  vector  m  the  matrix  T  Image 
vectors  x  can  be  mapped  into  the  final  feature  space  j : 

f  =  Tx.  (2) 

The  KY  transform  generates  features  which  are  more  dis¬ 
criminative.  The  eigenvalues,  generated  from  the  principal 
component  analysis  of  the  correlation  matrix,  are  usually  com¬ 
posed  of  two  parts  that  are  proportional  to  class  means  and 
class  variances,  respectively.  The  effects  of  these  two  parts  on 
the  feature's  discriminative  power  are  quite  opposite  [5].  The 
conflict  of  the  class  means  and  variances  in  the  eigenvalues, 
which  is  common  in  the  other  algorithms,  is  overcome  in  the 
KY  transform  as  follows.  The  contribution  of  the  class  vari¬ 
ance  in  / space  to  the  new  eigenvalues  r,  (k  =  1.  2.  3.  ...  , 
KM)  of  the  second  principal  component  analysis  remains  the  , 
same  as  those  in  the  v  space.  Therfore,  the  eigenvalues  rk  are 
solely  related  to  the  class  means: 

K 

n  =  1  +  2  P;/i,  k  =  1,  2,  3 . KM.  (3) 


yA,  is  the  Ath  component  of  the  ith  class  mean  in  /  space.  On 
the  other  hand,  the  feature  selection  criterion  <>,  [k  =  1.  2, 

3 . KM)  is  defined  as  the  weighted  summation  of  the 

average  mutual  distance  between  the  class  means: 

<5.  =  2  P.  t  P,(y.K,  ~  /.,)'>  k  =  1.  2.  3 . KM. 

• f  i  tr * 

(4) 

and  can  be  reduced  [5]  to 

<>,  =  r,  -  1.  (5) 

As  a  result,  the  feature  selection  criterion  can  be  based  on 
the  eigenvalues  rk.  which  are  solely  related  to  the  class  means. 
Therefore,  with  the  eigenvalues  n  arranged  in  descending 


Figure  1  Joint  transform  correlator  consisting  of  LCTV  liquid  crys¬ 
tal  television.  LI  transform  lens.  LCLV  liquid  crystal  light  valve.  BS 
beamsplitter,  and  L2  transform  lens 


order,  image  classification  which  uses  the  first  m  features 
(where  m  <  KM)  is  likely  to  be  more  accurate  than  the  other 
algorithms.  Moreover,  a  higher  SBP  process  in  image  clas¬ 
sification  is  possible  simply  because  the  classification  error 
rate  is  proportional  to  the  size  of  images  N  [10]. 

The  implementation  of  the  KY  transform  is  conducted  on 
a  programmable  JTC.  shown  in  Figure  1.  for  a  two-category 
image  classification.  In  the  training  process.  10  128  x  128 
training  images  (M  =  10)  of  each  class  are  used.  In  order  to 
limit  the  bias  between  the  training-image  error  rate  and  the 
test-image  error  rate  [11].  the  first  two  features  of  the  KY 
transform./,  and /,.  are  selected  for  the  classification  (m  = 
2).  In  the  two-dimensional  implementation  on  the  JTC.  the 
feature  extraction  can  be  represented  by  the  following: 

/<  =  i  i  k  =  1.2 . m.  (6) 

a-  I  ft*  l 


where  tlnl,  is  the  n  x  n  array  format  of  the  column  vector  tt 
in  Eq.  (2)  and  x ^  is  the  input  image.  The  three-dimensional 
KY  transform  matrix  T  can  be  partitioned  into  m  submatrices 
which  are  called  the  KY  basis  images.  The  basis  images 
1  and  2  are  shown  in  Figure  2(a)  and  (b).  respectively.  The 
KY  features /*  of  a  target  image  are  obtained  at  the  correlation 
center  in  the  output  plane  of  the  JTC.  while  the  target  and 
the  kih  KY  basis  image  are  displayed,  side  by  side,  in  the 
input  plane.  A  3-in  liquid  crystal  television  (LCTV)  of 
220  x  240  pixels  is  used  as  the  input  device,  which  is  sub¬ 
merged  in  a  liquid  gate  to  make  the  phase  distribution  across 
the  aperture  uniform.  A  48-in  lens  has  to  be  used  as  the  front 
Fourier  transform  lens  in  order  to  write  the  essential  grating 
in  the  joint  power  spectrum  on  the  liquid  crystal  light  valve 
(LCLV).  which  has  a  cutoff  frequency  of  40  lines/mm.  The 
shift  invariant  property  of  the  JTC  cannot  be  fully  utilized, 
due  to  the  limited  number  of  displaying  elements  in  the  LCTV. 
The  LCTV  also  affects  the  contrast  ratio  of  the  JTC.  However, 
by  reducing  the  frequency  of  the  ac  power  supply  for  the 
LCLV  to  1  kHz.  the  dc  spot  in  the  joint  power  spectrum 


(a) 


(b) 


Figure  2  (a)  The  KY  basis  image  1.  and  (b)  basis  image  2 
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diminishes  so  that  the  light  leakage  through  the  space  between 
the  electrodes  is  blocked  at  the  frequency  plane.  Therefore, 
a  contrast  ratio  of  30  :  1  is  achieved  in  the  JTC.  The  addressing 
and  relaxation  times  of  the  JTC  are  80  and  70  ms,  respectively, 
of  which  the  LCLV  contributes  a  major  part.  A  television 
frame  rate  (30  frames/sec)  operation  can  be  achieved  by  re¬ 
placing  the  CdS  LCLV  with  the  Si  LCLV  [12]. 

Increased  cross-coiTelations  are  obtained  in  the  output 
plane  by  taking  advantage  of  the  digitizing  effect  of  the  LCTV. 
Due  to  the  digitization  of  the  input  image  by  the  LCTV,  the 
joint  power  spectrum  takes  a  lattice  form,  where  each  dot  in 
the  lattice  carries  the  same  information  content.  The  cross 
correlation  between  a  target  and  a  KY  basis  image  in  the 
output  plane  is  given  by 

o(x.  y)  =  {[/(at .  y)  0  b(x,  y)]  *  <5(.t  -  2£sx.  y)} 

x  |  2  exp[-t'2;r(m'.r/5,  +  rt'y/s,)]j,  (7) 

where  the  symbols  0  and  4  denote  the  two-dimensional  cor¬ 
relation  and  convolution  operation,  respectively;  t(x.  y )  and 
b( x.  y)  represent  the  target  and  the  KY  basis  image,  respec¬ 
tively;  s,  and  s,  are  the  sampling  periods  of  the  LCTV  in  the 
x  and  v  directions:  2Ax  is  the  separation  between  the  two 
images;  A  is  the  wavelength  of  the  readout  coherent  light:  and 
/is  the  focal  length  of  the  lens  L2.  By  properly  adjusting  the 
iris  in  front  of  the  LCLV.  the  magnitude  of  the  extra  phase 
term  in  Eq.  (7)  may  exceed  unity  such  that  substantially  in¬ 
creased  correlation  peaks  can  be  obtained.  An  example  of 
the  correlation  output  is  shown  in  Figure  3,  where  zero-order 
diffraction  (i.e..  dc)  is  partially  blocked.  Moreover,  a  quasi- 


monochromatic  and  partial  coherent  can  be  employed  in  read¬ 
out  to  reduce  the  coherent  noise. 

The  two  KY  features,  /,  and  f2,  are  extracted  from  a  target 
image  m  two  separate  joint  transform  correlation  operations. 
The  same  process  is  repeated  32  times  to  generate  the  two 
KY  features  for  all  32  images  (20  of  which  are  sample  images, 
and  12  of  which  are  test  images).  The  KY  features  of  alt  the 
images  are  plotted  in  Figure  4.  A  solution  linear  classifier  is 
obtained  by  using  the  gradient  descent  method  [13]  and  is 
also  plotted  in  Figure  4.  The  results  show  that  there  is  no 
error  in  the  two-category  classification  of  the  32  128  x  128 
images.  Also,  the  image  classification  by  the  KY  transform 
can  be  easily  extended  to  multicategory  cases. 
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Neural  network  model  using  interpattern  association 


Taiwei  Lu,  Xin  Xu,  Sudong  Wu,  and  Francis  T.  S.  Yu 


This  paper  investigates  a  neural  network  model — interpattem  association  (IP A)  model — in  which  the  basic 
logical  operations  are  used  to  determine  the  interpattem  association  (i.e.,  association  between  the  reference 
patterns),  and  simple  logical  rules  are  applied  to  construct  tristate  interconnections  in  the  network.  Comput¬ 
er  simulations  for  the  reconstruction  of  similar  English  letters  embedded  in  the  random  noise  by  the  IPA 
model  have  shown  improved  performance  compared  with  the  Hopfield  model  A  2-D  hybrid  optical  neural 
network  is  used  to  demonstrate  the  usefulness  of  the  IPA  model.  Since  there  are  only  three  gray  levels  used  in 
the  interconnection  weight  matrix  for  the  IPA  model  the  dynamic  range  imposed  on  a  spatial  light  modulator 
is  rather  relaxed,  and  the  interconnections  are  much  simpler  than  the  Hopfield  model 


I 

e 
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I.  Introduction 

Neural  networks  have  shown  effectiveness  in  pat¬ 
tern  recognition.1'  For  recognition  of  a  given  set  of 
reference  patterns,  there  are  two  schemes  to  construct 
the  associative  memory  matrix  (i.e.,  interconnection 
weight  matrix).  The  first  is  the  so-called  intrapattern 
association,  which  emphasizes  the  association  of  ele¬ 
ments  within  each  reference  pattern.  For  example,  in 
the  Hopfield  model,  the  outer  products  of  the  refer¬ 
ence  patterns  are  added  to  form  the  interconnection 
weight  matrix.  This  type  of  approach  may  create  an 
unstable  or  ill-conditioned  network  if  the  reference 
patterns  are  not  independent  of  each  other.  The  sec¬ 
ond  scheme  is  an  interpattem  association  (IPA)  for 
which  the  interconnection  weight  matrix  is  construct¬ 
ed  by  smphasizbig  the  association  between  reference 
patterns.  Where  tt?  reference  patterns  are  similar  to 
each  other  (e.g.,  human  faces,  fingerprints,  handwrit¬ 
ten  characters),  the  special  features  of  each  pattern 
become  very  important  in  pattern  recognition. 
Therefore,  it  is  necessary  to  consider  the  relationships 
between  the  special  and  common  features  among  the 
reference  patterns  in  constructing  the  interconnection 
weight  matrix. 

We  present  a  neural  network  model  based  on  the 
association  between  reference  patterns.  This  model 
first  determines  the  common  and  special  features 
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among  the  reference  patterns  by  applying  a  logical 
operation.  Two  equivalent  logical  rules  are  presented 
in  Sec.  II  to  construct  the  excitatory  and  inhibitory 
interconnections  in  the  network.  In  Sec.  Ill,  the  IPA 
model  is  compared  with  the  Hopfield  model  Com¬ 
puter  simulations  for  the  reconstruction  of  similar  En¬ 
glish  letters  in  the  random  noise  have  shown  that  the 
IPA  model  performs  better  than  the  Hopfield  model. 
In  Sec.  IV,  a  2-D  hybrid  optical  neural  network  is  used 
to  implement  this  new  model.  The  experimental  re¬ 
sults  show  that  the  IPA  model  indeed  performs  more 
effectively  than  the  Hopfield  model. 

II.  Interpattem  Association  Model 

The  IPA  is  important  in  the  construction  of  the 
associative  memory  matrix.  The  information  ob¬ 
tained  from  special  areas  (i.e.,  areas  occupied  by  one 
pattern)  is  important  than  from  common  or  over¬ 
lapped  areas. 

Figure  1  shows  an  example  of  a  reference  set  that 
consists  of  three  overlapping  patterns  A,  B,  and  C  in 
the  pattern  space.  These  patterns  can  be  divided  into 
seven  subbpaces,  I,  II,  and  III  are  the  special  areas  of 
patterns  A,  B,  and  C,  respectively.  IV,  V,  and  N’T  are 
the  common  areas  of  A  and  B:  B  and  C,  C  and  A, 
respectively.  VII  is  the  common  area  of  A,  B.  and  C. 
The  rest  can  be  defined  as  an  empty  space  These 
areas  can  be  expressed  by  the  following  logic  functions: 

I  =  A  a  (B  v  c), 

II  =«  B  a  (aTC),  V  =*  (B  a  C)  A  A, 

in  =  c  a  (aTb),  vi  =  (c  a  a)  a  B, 

17  =  (A  A  B)  A  5,  VII  -  (A  a  B  a  C)  a  5,  (D 

where  a  ,  v ,  and  stand  for  the  logic  AND,  OR,  and 
NOT  operations,  respectively. 
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Fig.  1,  Common  and  special  areas  of  three  reference  patterns. 


(a)  (b)  (c) 


Fig.  2.  Example  of  constructing  an  PA  neural  network:  Three  2  X 
2  reference  patterns:  A  (a),  B  (b),  C  (c),  tristate  neural  network  (d), 
interconnection  weight  matrix  (e). 


With  the  above  information,  we  can  start  building 
the  interconnections  for  a  single  layer  neural  network. 
To  illustrate  the  concept  of  IPA  in  this  example,  each 
pixel  in  the  reference  pattern  space  is  mapped  to  each 
input/'mtput  neuron. 

When  an  input  neuron  in  area  VII  is  on  (i.e.,  the 
input  of  this  neuron  is  1),  it  implies  that  there  is  an 
input  but  not  whether  it  belongs  to  any  pattern.  Thus 
this  neuron  can  only  excite  the  output  neurons  within 
area  VII  and  have  no  connection  with  the  output  neu¬ 
rons  in  other  areas. 

When  an  input  neuron  in  area  V  is  on,  it  implies  that 
the  input  pattern  is  not  A  but  not  whether  it  belongs  to 
pattern  B  or  C.  Thus  this  neuron  must  excite  the 
output  neurons  in  areas  V  and  VII  but  inhibit  the 
output  neurons  in  area  I,  similarly  for  input  neurons  in 
area  IV  of  VI. 

For  the  case  when  an  input  neuron  is  on  in  area  I,  it 
implies  that  pattern  A  appears  at  the  input  end. 
Therefore,  this  neuron  can  excite  all  the  output  neu¬ 
rons  in  pattern  A  (i.e.,  areas  I,  IV,  VI,  and  VI_I)  but 
inhibit  the  output  neurons  in  areas  ( B  v  C)  a  A  (i.e., 
areas  II,  III,  and  V).  Similarly,  for  the  input  neurons 
in  area  II  or  III,  they  must  excite  output  neurons  in 


areaB  or  C  and  inhibit  neurons  in  areas  I,  III,  and  VI  or 
areas  I,  II,  and  IV,  respectively. 

In  view  of  Eq.  (1),  the  descriptive  logic  can  be  sum¬ 
marized  in  the  following: 

Rule  A: 

Assume  that  a  subarea  X  is  represented  by  the  fol- 


lowing  expression: 

X  =  P  A 

(2) 

where 

P  *  Pi  a  p2  a  . . .  A  p„. 

(3) 

Q  =  9i  v  q2  v  ...  v9n. 

(4) 

Here  pu  p2, . . . ,  pn  and  q  i,  q2,...,qm  are  reference 
patterns,  n  and  m  are  two  positive  integers,  and  n  +  m 
-  M  is  the  total  number  of  reference  patterns.  The 
input  neurons  in  area  X  must  excite  (i.e.,  having  posi¬ 
tive  connections  with)  all  the  output  neurons  in  areaP, 
inhibit  (i.e.,  having  negative  connections  with)  all  out¬ 
put  neurons  in  area  Q  A  F  where  F  is  defined  by 

F  =  Pi  vpi  v  ...  vp„,  (5) 

and  have  no  connection  with  the  output  neurons  in  the 
remaining  areas.  For  simplicity,  the  connection 
strengths  (i.e.,  weights)  are  defined  as  1  for  positive 
connections,  -1  for  negative  connections,  and  0  for  no 
connection.  Thus  the  IPA  neural  network  can  be  con¬ 
structed  in  a  simple  three-state  structure. 

To  illustrate  further  the  construction  of  the  IPA 
model,  A,  B,  and  C  are  assumed  as  three  2X2  array 
patterns  shown  in  Figs.  2(a),  (b),  and  (c),  respectively. 
The  pixel-pattern  relationship  is  given  in  Table  I.  It 
is  apparent  that  pixel  1  represents  the  common  feature 
of  A,  B,  and  C,  pixel  2  is  the  common  feature  of  A  and  B, 
pixel  3  is  the  common  feature  of  A  and  C,  and  pixel  4 
represents  the  special  feature  of  C. 

By  applying  Rule  A  to  the  three  reference  patterns 
A,  B,  and  C,  a  tristate  neural  network  can  be  construct¬ 
ed,  as  illustrated  in  Fig.  2(d).  This  is  a  one-layer 
neural  network  with  four  input  neurons  and  four  out¬ 
put  neurons.  Each  neuron  is  matched  to  one  pixel  of 
the  reference  patterns.  For  example,  the  first  input 
neurons  corresponds  to  pixel  1  of  the  input  pattern  and 
excites  only  the  first  output  neuron.  The  second  input 
neuron  (the  corresponding  pixel  belongs  to  patterns  A 
and  B)  excites  both  the  first  and  second  output  neu- 


Table  L  Pixel-Pattern  Relalionthlp  of  Three  Reference 
Pattern* 
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rons,  while  inhibiting  the  fourth  output  neuron,  which 
belongs  to  the  special  area  of  pattern  C. 

It  is  interesting  to  analyze  the  structure  of  the  I WM. 
Since  the  reference  patterns  are  in  2-D  form,  the 
weight  matrix  becomes  a  4-D  matrix.  We  can  parti¬ 
tion  the  4-D  IWM  into  a  2-D  submatrix  array,5  as 
illustrated  in  Fig.  2(e).  The  IWM  can  be  divided  into 
foui-  blocks.  Each  block  corresponds  to  one  output 
neuron.  The  four  elements  in  one  block  represent  the 
four  neurons  in  the  input  end.  For  example,  since  all 
four  elements  in  the  upper  left  block  have  a  value  of  i, 
it  indicates  that  either  one  of  the  four  input  neurons 
can  excite  the  first  output  neuron.  In  the  upper  right 
block,  as  another  example,  the  first  and  third  elements 
are  0,  the  second  element  has  a  value  of  1,  and  the 
fourth  element  is  —1.  From  this  example  we  can  de¬ 
termine  that  the  first  and  third  hput  neurons  have  no 
connect’or  to  me  second  output  neuron,  the  second 
input  neuron  excites  the  second  output  neuron,  and 
the  fourth  input  neuron  inhibits  the  second  output 
neuron. 

To  simplify  the  algebraic  operations,  an  equivalent 
rule,  Rule  B,  is  developed  as  follows.  It  can  be  used  to 
construct  the  IWM  by  examining  the  pix^l-pattern 
relationships. 

Rule  B. 

Let  us  define  Du  as  a  2-D  matrix  that  corresponds  to 
the  2-D  array  in  Table  I,  where  l  and  i  denote  the  row 
and  column  numbers.  Let  d,  be  the  number  of  pat¬ 
terns  that  are  bright  (i.e.,  in  state  1)  at  the  ith  pixel, 
then  it  can  be  determined  by  summing  the  elements  in 
the  ith  column  of  Table  I,  i.e., 

M 

(6) 

i-i 

Let  us  also  define 

M 

Kj  “  X  (7) 

;-i 

which  is  the  sum  of  the  product  of  columns  i  and  j  in 
Table  I.  Ther.  we  can  construct  an  IPA  neural  net¬ 
work  by  apph  ing  the  following  logical  rules: 

(1)  If  kij  =  min  (di,d;), 

when  di  <  dj,  pixel  i  must  excite  pixel  j,  but  pixel  j 
must  not  excite  pixel  i; 

when  di  =  dj,  pixels  i  and  j  must  excite  each  other: 

when  d(  >  d,,  pixel  j  must  excite  pixel  i,  but  pixel  i 
must  not  excite  pixel  j; 

(2}  If'  0  <  <  min  (d,-,d;),  pixels  i  and  j  have  no 

connection  to  each  other. 

(3)  If  =  0, 

when  d{  ^  0,  and  dj  ^  0,  pixels  i  and  j  must  inhibit 
each  other; 

when  d,  —  0  and/or  d,  =  0,  pixels  i  and  j  must  have  no 
connection  to  each  other. 

We  stress  that  these  rules  for  IWMs  are  simple  and 
straightforward,  which  is  suitable  for  computer  calcu¬ 
lations.  In  fact,  the  IWMs  computed  by  Rule  B  are 
equivalent  to  those  obtained  by  Rule  A. 


Comparison  with  the  Hopfield  Model 

Tt  is  the  differences  rather  than  the  similarities 
among  patterns  that  are  used  for  pattern  recognition. 
For  example,  the  outline  of  the  eyes,  nose,  and  mouth 
are  common  features  in  all  human  faces.  People  distin¬ 
guish  different  persons  by  the  differences  rather  thar. 
the  detail  in  the  faces. 

Similar  to  some  other  neural  network  algorithms, 
the  Hopfield  model  constructs  the  IWM  by  correlating 
the  elements  within  each  pattern,  however,  ignoring 
the  relationships  among  the  reference  patterns.  The 
IWM  T  of  the  Hopfield  model  for  three  reference 
patterns  A,  B,  and  C  can  generally  be  expressed  as 

T  =  AAt  +  BBt  +  CCt-  31,  (8) 

where  T  is  the  transpose  of  the  vectors  and  I  is  the  unit 
matrix,  which  makes  the  weight  matrix  zero  diagonal. 

If  input  pattern  A  is  applied  ti>  the  neural  system, 
the  output  would  be 

V  =  TA 

*  A(ATA)  +  B(BTA)  +  C(CTA)  -  3 A,  (9) 

where  ATA  represents  the  autocorrelation  of  pattern 
A,  while  BTA  and  CTA  are  the  respective  cross-correla¬ 
tion  between  A  and  B,  and  A  and  C.  If  the  differences 
between  A,  B  and  C  are  sufficiently  large,  the  autocor¬ 
relation  of  A,  B  or  C  would  be  much  larger  than  the 
cross-correlation  between  them,  i.e., 

ATA » BTA,  ATA»CTA.  (10) 

Notice  that  from  Eq.  (S)  pattern  A  has  a  larger  weight¬ 
ing  factor  than  patterns  3  and  C,  for  which  patterns  B 
and  C  can  be  considered  noise.  By  choosing  the  prop¬ 
er  threshold  value,  pattern  A  will  be  constructed  at  the 
output  end  of  the  neural  network. 

On  the  other  hand,  if  A,  B,  and  C  are  very  similar,  Eq. 
(10)  no  longer  holds;  since  the  weights  of  patterns  B 
and  C  are  comparable  with  those  of  A,  patterns  B  and  C 
can  no  longer  be  considered  noise.  Thus  the  threshold 
value  for  the  Hopfie'd  model  cannot  be  easily  defined; 
the  Hopfield  mode'  thus  becomes  unstable. 

Computer  simulations  of  a  2-D  neural  network  with 
8X8  array  neurons  at  the  input  and  output  ends  have 
been  conducted  for  both  Hopfield  and  IPA  models. 
The  reference  patterns  considered  are  the  twenty-six 
capital  English  letters  l5*.  ;•  :n  sequence  based  on 

the  similarities  of  the  ,  r3,r,R,F, . . . ,  and  each 
letter  occupies  an  8  X  $  array  pixel. 

Figure  3  shows  the  error  rates  as  a  function  of  refer¬ 
ence  patterns  for  Hoc  c.ald  and  IPA  models  with  noisy 
inputs.  The  input  SNR  for  computer  simulation  is  ~7 
dB.  We  notice  that  the  Hopfield  model  becomes  un¬ 
stable  when  patterns  B,  P,  and  R  are  stored  in  the 
IWM,  whereas  the  IPA  model  is  quite  stable  with 
twelve  stored  letters.  Comparison  is  also  made  for 
noiseless  input.  In  this  condition,  the  IPA  model  can 
produce  correct  results  for  all  twenty-six  stored  letters 
in  the  IWM,  whereas  the  Hopfield  model  starts  making 
significant  errors  wh-'p  the  number  reference  patterns 
are  increased  tn  3. 
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Number  ol  refen  ice  patterns 
Input  SNR  =7oB  otal  pixel  No.  «=  64 

Fig.  3.  Comparison  of  the  IP/  And  Hopfield  models. 


IV.  Optical  Implementation 

The  2-D  and  3-D  parallel  processing  capabilities  of 
the  optical  system  make  it  ac  important  candidate  in 
solving  the  complex  intercoms -ctions  of  a  neural  net¬ 
work.5-68 

An  optical  neural  network6  is  used  for  implementing 
the  IPA  model  The  optical  art  hitecture  is  illustrated 
in  Fig.  4.  In  this  system,  a  high  resolution  video  moni¬ 
tor  is  used  to  display  the  IWM,  which  also  provides  the 
source  of  incoherent  light  for  processing.  This  system 
differs  from  the  matrix-vector  processor,  as  proposed 
by  Farhat  et  al ,5  in  which  the  positions  of  the  input 
array  and  weighting  matrix  mask  have  been  ex¬ 
changed.  Notice  that  this  arrangement  makes  it  pos¬ 
sible  to  use  a  video  monitor  for  IWI.1  instead  of  a  low 
resolution,  low  contrast  SLM.  A  lenrlet  array  consist¬ 
ing  of  N  X  N  lenses  is  used  to  establish  the  optical 
interconnections.  A  moderate  sized  programmable 
SLM  of  NXN  binary  pixels  serves  as  the  input  device. 
The  light  beam  emanating  from  each  block  of  the  TV 
screen  (i.e.,  submatrix  of  IWM)  is  passed  through  a 
specific  lens  of  the  lenslet  array  and  is  imaged  onto  the 
input  SLM.  The  beams  passing  the  SLM  are  then 
imaged  oato  the  output  plane  by  an  imaging  lens  to 
form  an  N  X  N  array  output  image  which  represents 
the  product  of  a  4-D  IWM  with  a  2-D  input  array.  The 
output  signals  can  be  picked  up  by  an  N  X  N  array 
photodetector. 

To  construct  a  closed  loop  neural  nexwork,  the  out¬ 
put  signals  from  the  detector  array  are  fed  back  to  the 
input  SLM  via  a  threshold  circuit  By  using  the  com¬ 
puter  to  calculate  and  modify  the  IWM,  the  optical 
neural  network  can  be  made  both  adaptive  and  pro¬ 
grammable. 

In  experiments,  Hopfield  and  IPA  models  are  cho¬ 
sen  to  perform  pattern  recognition  using  noisy  input 
patterns.  B,  P,  and  R  are  three  lexers  stored  in  the 
weight  matrix  as  shown  in  FI?,  ^a).  The  positive  and 
negative  parts  of  the  T'.VMs  for  the  IPA  model  are 
shown  in  Figs.  5(l>i  and  (ch  while  those  for  the  Hop- 
field  model  are  displayed  in  Figs.  5(d)  and  (e).  By 
comparing  these  two  IWM  sets  it  can  he  seen  that  the 
IPA  model  has  two  major  advantages  over  the  Hop- 
field  model,  namely,  (1)  ‘‘ev.^r  interconnections  and  (2) 
fewer  gray  levels.  Tr.*>  latter  is  significant  because  the 


vid.0  Monitor  L»n$l«t  A  fty  lrn,ging 


Fig.  4.  Two-dir  ..  ■  j:  optical  neural  network. 


Fig.  5.  Experiments  on  sr  optical  neurocomputar.  Three  similar 
English  iotten  as  reference  patterns  (a),  positive  and  negative  part* 
of  the  IWMs  of  the  IPA  model  (b)  and  (c)  and  the  Hopfield  model  (d) 
and  (ev,  input  pattern,  SNR  =  7  dB  (0;  results  of  pattern  reconstruc¬ 
tion  by  using  IPA  model  (g)  and  Hopfieid  model  (h). 


IPA  model  requires  only  three  gray  levels  to  repreent 
the  IWM,  whereas  the  Hopfield  model  needs  2 M  +  I 
gray  levels,  where  M  denotes  the  number  of  stored 
reference  patterns. 

The  experimental  results  of  these  two  models  are 
obtained  based  on  an  input  pattern  B  embedded  in 
30%  random  noise  (SNR  -  7  dB),  as  shown  in  Fig.  5(f). 
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The  output  patterns  of  the  IPA  and  Hopfield  models 
are  shown  in  Figs.  5(g)  and  (h),  respectively.  Because 
of  the  curvature  of  the  video  monitor  screen,  the  out¬ 
put  results  are  somewhat  distorted.  Nevertheless,  the 
results  from  the  IPA  model  have  been  shown  to  be 
better  than  those  from  the  Hopfield  model. 

V.  Conclusions 

We  have  proposed  a  new  model— an  IF  A  neural 
network.  By  using  a  simple  logical  rule,  the  common 
and  special  features  of  the  reference  patterns  can  be 
obtained,  and  the  positive  or  negative  interconnec¬ 
tions  can  be  assigned  to  each  neuron.  The  IWM  can  be 
easily  formulated ,  which  requires  merely  an  SLM  with 
three  gray  levels  to  realize  the  IWM.  Computer  simu¬ 
lations  and  experiments  reveal  that  the  IPA  model  is 
more  effective  in  performing  pattern  recognition 
(among  similar  patterns)  than  the  models  that  deal 
only  with  the  intrapattern  association. 

We  acknowledge  the  support  by  U.S.  Army  Missile 
Command  through  the  U.S.  Army  Research  Office 
under  contract  DAAL03-87-K-0147. 
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A  compact  optical  neural  network  that  uses  high-resolution  liquid-crystal  televisions  has  been  constructed.  System 
design  considerations  and  an  experimental  demonstration  of  the  liquid-crystal  television  neural  network  are 
reported. 


Two  of  the  most  important  features  of  an  artificial 
neural  network  must  be  the  possibility  of  massive  in¬ 
terconnections  and  parallel-processing  operations.1’2 
Recently,  the  use  of  optical  techniques  to  perform  the 
massive  interconnection  in  a  two-dimensional  optical 
neural  network  was  suggested  by  Farhat  and  Psaltis,3 
for  which  they  used  the  basic  concept  of  a  vector- 
matrix  optical  processor.  To  provide  the  network 
with  self-organizing  and  learning  capabilities,  a  high- 
resolution,  large-dynamic-range,  programi  .able  spa¬ 
tial  light  modulator  is  required  for  the  interconnection 
weight  matrix  (IWM)  construction.  However,  cur¬ 
rently  available  spatial  light  modulators  have  small 
space-bandwidth  products  with  limited  gray  levels, 
making  it  difficult  actually  to  build  such  an  optical 
neural  network. 

We  recently  proposed  an  optical  architecture  using 
a  video  monitor  as  a  programmable  IWM  for  the  im¬ 
plementation  of  optical  neural  networks.4  5  The  reso¬ 
lution  of  the  video  monitor  is  approximately  1000  X 
1000  pixels,  and  the  dynamic  range  is  approximately 
256  gray  levels.  In  fact,  an  8  X  8  neuron  optical  neural 
network  was  built  recently  in  our  laboratory.  Howev¬ 
er,  the  physical  size  of  a  video  monitor  makes  the 
system  large,  and  the  curvature  of  the  monitor  screen 
also  poses  some  problems.  To  alleviate  these  two  ma¬ 
jor  problems,  in  this  Letter  we  present  a  compact  opti¬ 
cal  neural  network  using  pocket-sized  liquid-crystal 
televisions  (LCTV’s). 

Generally,  an  N -neuron  network  requires  up  to  N2 
interconnections.  As  Hopfieid  suggested,  the  state  of 
each  neuron  may  be  expressed  by  the  following  itera¬ 
tive  equation: 


v,(n  +  1)  =  f 


i  =  (1) 


where  v,  is  the  state  of  the  ith  neuron,  n  is  for  the  nth 
iteration,  T,,  represents  the  connection  strength  be¬ 
tween  the  ;th  and  the  ith  neurons,  and  f  is  a  nonlinear 
operator  of  the  neurons. 


As  an  example,  if  the  neurons  operate  in  bipolar 
states,  f  can  be  described  as 


x  >  t 
x<t' 


(2) 


where  t  is  the  thresholding  value.  Note  that  the  func¬ 
tional  operation  of  Eq.  (1)  is  a  nonlinear  operator  that 
depends  on  the  matrix-vector  product,  which  can  be 
expressed  in  a  one-dimensional  vector  representation, 
such  as 


V(n  +  1)  =  f[TV(n)], 


(3) 


where  V  is  the  state  vector  of  the  neurons  and  T  is  the 
IWM,  which  is  also  called  the  associative  memory  of 
the  neural  network. 

It  is  apparent  that,  for  a  two-dimensional  N  x  N 
neuron  network,  the  iterative  equation  can  be  extend¬ 
ed  in  the  form 


‘  .v  .v 

u,k(n  +  1)  =  f  VY  TlklJv,(n)  ,  (4) 

_i=l  ;*>1 

where  u;*  represents  the  state  of  the  Ik th  neuron  in  an 
NX  N  space  and  Tiki,  is  a  four-dimensional  intercon¬ 
nection  weight  matrix.  Note  that  the  matrix  T  can  be 
partitioned  into  an  array  of  two-dimensional  subma¬ 
trices  Tn.,,  TIft„  . . .  T,sn,,,  in  which  each  submatrix  is 
an  NxN  size.3C  Thus  we  see  that  a  four-dimensional 
associative  memory  matrix  can  be  displayed  as  an  N2 
X  N 2  two-dimensional  array  representation  (Fig.  1). 

In  recent  developments  of  the  LCTV,  the  imaging 
quality  has  been  reported  to  be  close  to  that  of  the 
commercially  available  high-resolution  video  moni¬ 
tors.  For  instance,  the  contrast  ratio  of  a  recent 
LCTV,  with  built-in  thin-film  transistors,  can  be  high¬ 
er  than  30:1,  and  the  dynamic  range  is  approximately 
16  gray  levels,  which  can  be  continuously  adjusted." 
The  optical  architecture  that  we  are  presenting  in¬ 
cludes  a  Hitachi  Model  C5-LC1  5  in.  (12.7-cm)  color 
LCTV  that  is  used  to  display  the  IWM  and  a  Seiko 
Model  LVD-202  2.7-in.  (6.9-cm)  color  LCTV  that  is 
used  to  display  the  input  patterns.  The  resolutions  of 
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Fig.  1.  Partition  of  a  four-dimensional  matrix  Tit,,,  into  an 
array  of  two-dimensional  submatrices  Tn,,,  Tm,,  -  ■  ■  TNNlJ. 
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Fig.  2.  Schematic  of  the  compact  optical  neural  network. 


back  to  the  LCTV2  for  a  second  iteration.  Note  that 
the  data  flow  in  the  optical  system  is  controlled  pri¬ 
marily  by  a  microcomputer  (PC).  For  instance,  the 
IWM’s  and  the  input  patterns  can  be  written  onto  the 
LCTVl  and  the  LCTV2  through  the  PC,  and  the  PC 
can  also  make  decisions  based  on  the  output  results  of 
the  neural  network.  Thus  the  proposed  LCTV  optical 
neural  network  is  indeed  a  programmable  and  adap¬ 
tive  neural  network. 

In  the  experimental  setup  of  the  compact  optical 
neural  network,  an  80-W  xenon-arc  lamp  is  used  as  the 
light  source.  LCTVl  is  used  to  display  the  IWM.  An 
8X8  lenslet  array  of  plano-convex  lenses  (6-mm  diam¬ 
eter,  72-mm  focal  length)  sandwiched  between  two 
optical  flat  glass  plates  is  used  to  establish  the  optical 
interconnection  between  LCTVl  and  LCTV2;  LCTV2 
is  used  for  input  pattern  generation.  An  imaging  lens 
set  (two  identical  plano-convex  lenses  of  100-mm  di¬ 
ameter  and  125-mm  focal  length,  sealed  face  to  face  in 
a  mount,  with  an  effective  70-mm  focal  length)  is  used 
to  image  the  output  result  from  LCTV2  onto  the  CCD 
camera.  This  arrangement  of  plano-convex  lenses  has 


these  LCTV’s  are  256  X  420  pixels  and  220  X  330 
pixels,  respectively. 

A  diagram  of  a  compact  LCTV  optical  neural  net¬ 
work  is  shown  in  Fig.  2.  A  neural  network  with  8X8" 
64  fully  interconnected  neurons  was  recently  built  in 
our  laboratory,  for  which  an  80-W  xenon-arc  lamp  was 
used  as  the  incoherent  light  source.  In  the  present 
investigation  the  Hitachi  LCTV  (LCTVl)  is  used  for 
the  generation  of  the  IWM,  which  consists  of  an  8  X  8 
array  of  submatrices,  with  each  submatrix  having  8  X 
8  elements.  This  IWM  is  then  displayed  on  a  fine 
diffuser  immediately  behind  LCTVl.  The  Seiko 
LCTV  (LCTV2)  is  used  as  an  input  device  for  the 
generation  of  the  input  patterns.  The  lenslet  array, 
which  consists  of  8  X  8  lenses,  provides  the  intercon¬ 
nections  between  the  IWM  and  the  input  pattern. 
Each  lens  of  the  lenslet  array  images  each  of  the  IWM 
submatriceb  onto  the  input  LCTV2  to  establish  the 
proper  interconnections.  Thus  we  see  that  the  input 
matrix  (i.e.,  LCTV2)  is  superimposed  onto  and  multi¬ 
plied  (or  interconnected)  by  all  the  IWM  submatrices. 
This  represents  the  connective  part  of  Eq.  (1),  i.e., 

■«1  I 

The  output  result  from  LCTV2  is  then  imaged  onto  a 
charge-coupled-device  (CCD)  array  detector.  The 
signals  collected  by  the  CCD  camera  are  then  sent  to  a 
thresholding  circuit,  and  the  final  results  can  be  fed 


Fig.  3.  Experimental  results  of  the  compact  optical  neural 
network,  (a)  Four  cartoon  images,  a  tree,  a  dog,  a  house,  and 
an  aircraft,  used  as  reference  patterns,  (b)  Positive  and  (c) 
negative  parts  of  the  IWM  generated  by  using  the  interpat¬ 
tern  association  model,  (d)  Input  pattern  embedded  in  25% 
noise,  and  (e)  its  output  result:  a  dog,  obtained  in  one 
iteration.  (0  Input  pattern  in  25%  noise,  and  (g)  its  output 
result:  a  tree,  obtained  in  one  iteration. 
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the  ability  to  correct  lens  distortion.  At  the  end  of  the 
system  a  Sony  CCD  camera  (Model  XC-77)  is  used  as 
an  output  array  detector. 

The  size  of  the  optical  neural  network  is  approxi¬ 
mately  65  cm  X  12  cm  X  12  cm.  and  the  size  of  LCTVl 
is  approximately  12  cm.  If  a  smaller  LCTV  is  used  for 
IWM  generation  (e.g.,  a  7-cm  Epson  color  LCTV), 
then  the  system  size  can  be  further  reduced  to  approx¬ 
imately  20  cm  X  7  cm  X  7  cm.  In  this  case,  the 
diameter  of  the  lenses  in  a  16  X  20  lenslet  array  is 
within  the  submillimeter  range,  for  which  the  lenslet 
array  can  be  synthesized  by  using  binary  optics5  or 
holographic  techniques. 

Since  the  LCTVl  has  a  resolution  of  256  X  420 
pixels,  a  maximum  number  of  16  X  20  =  320  neurons 
can  actually  be  built.  The  matrix-vector  operations 
of  the  proposed  optical  neural  network  are  performed 
in  parallel.  The  processing  speed  is  limited  by  the 
speed  of  the  LCTV’s,  which  have  an  addressing  speed 
of  30  frames/sec.  Thus  the  operational  speed  of  the 
proposed  neural  network  with  8X8  neurons  is  approx¬ 
imately  30(8  X  S)2  —  1.2  X  105  interconnection  opera¬ 
tions  per  second.  However,  for  a  16  X  20  neural  net¬ 
work  the  operational  speed  can  be  as  high  as  30(16  X 
20)2  =  3  X  106  interconnection  operations  per  second. 

In  experiments,  we  have  used  the  Hopfield,  orthogo¬ 
nal  projection,  and  interpattern  association'5  models 
for  the  generation  of  the  IWM’s.  One  of  the  interpattem 
association  experimental  demonstrations  is  shown  in 
Fig.  3.  Four  cartoon  images,  a  tree,  a  dog,  a  house,  and 
an  aircraft,  are  used  as  reference  patterns,  and  each 
letter  is  displayed  in  an  8  X  8  pixel  array  matrix  [Fig. 
3(a)].  We  have  used  the  interpattern  association 
model  (which  is  based  on  the  association  among  the 
reference  patterns9)  for  the  construction  of  the  IWM. 
The  positive  and  negative  parts  of  the  IWM  generated 
by  the  interpattem  association  model  are  shown  in 
Figs.  3(b)  and  3(c),  respectively.  There  are,  however, 
ways  to  represent  the  negative  values  in  the  IWM,  by 
adding  a  bias  into  the  IWM  or  by  using  an  area-encod¬ 
ing  method.  However,  for  simplicity,  the  positive  and 
negative  parts  of  the  IWM  are  sequentially  displayed 
on  the  LCTVl  in  our  experimental  demonstrations. 
An  input  pattern,  i.e.,  a  dog  embedded  in  25%  random 
noise,  is  presented  on  LCTV2  (Fig.  3(d)].  The  input 
pattern  is  then  multiplied  by  the  submatrices  of  the 
IWM.  using  the  lenslet  array.  The  positive  and  nega¬ 


tive  parts  of  the  output  results  are  captured  by  the 
COD  camera,  and  the  subtraction  and  thresholding 
operations  are  performed  by  the  PC,  for  which  the 
output  result  is  shown  in  Fig.  3(e).  Thus  a  recogniz¬ 
able  dog  is  reconstructed  with  only  one  iteration  in 
this  experiment.  Similarly,  when  a  tree  with  25% 
noise  was  presented  on  LCTVl  [Fig.  3(f)],  an  output 
pattern  of  tree  without  any  noise  is  obtained,  as  illus¬ 
trated  in  Fig.  3(g).  From  this  example,  we  can  see  that 
the  compact  optical  neural  network  is  capable  of  per¬ 
forming  pattern-recognition  and  image-reconstruc¬ 
tion  operations. 

In  conclusion,  we  have  shown  that  a  compact  optical 
neural  network  that  uses  high-resolution  LCTV's  can 
be  constructed.  The  processing  speed  of  this  neural 
network  can  be  as  high  as  3  X  10s  interconnection 
operations  per  second,  if  a  16  X  20  lenslet  array  is  used. 
To  illustrate  the  performance  of  the  LCTV  optical 
neural  network,  an  experimental  result  obtained  with 
the  interpretation  association  model  is  presented.  Fi¬ 
nally,  we  stress  that  low-cost,  compact  optical  neural 
networks  can  indeed  be  built  using  inexpensive 
LCTV’s. 

We  acknowledge  the  support  of  the  U.S.  Army  Mis¬ 
sile  Command  through  the  U.S.  Army  Research  Office 
under  contract  DAAL03-87-0147. 
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A  nonconventional  joint-transform  correlator  (NJTC)  is  discussed.  We  show  that  the  shift-invariant  property  of 
the  usual  joint-transform  arrangement  can  be  preserved.  The  advantages  of  the  NJTC  are  the  efficient  use  of  the 
light  source,  the  use  of  smaller  transform  lenses,  higher  conflation  peaks,  and  a  higher  carrier  fringe  frequency. 


A  conventional  joint-transform  correlator1-3  (JTC)  of¬ 
fers  the  advantages  of  the  avoidance  of  normal 
matched-filter  synthesis,  a  higher  space-bandwidth 
product,  a  lower  carrier  frequency,  a  higher  degree  of 
modulation,  and  suitability  for  real-time  implementa¬ 
tion.  However,  the  conventional  JTC  also  suffers  sev¬ 
eral  major  drawbacks,  e.g.,  the  inefficient  use  of  illumi¬ 
nation  light,  the  requirement  of  a  larger  transform 
lens,  a  more  stringent  spatial  coherent  requirement, 
and  the  overall  small  size  of  the  joint-transform  spec¬ 
trum.  In  this  Letter  we  describe  an  alternative  ap¬ 
proach  that  alleviates  some  of  these  limitations. 

The  quasi-Fourier  transformation  of  an  optical  sys¬ 
tem  as  defined  here  is  shown  in  Fig.  1.  We  assume 
that  the  detection  plane  P2  is  located  a  small  distance  5 
away  from  the  back  focal  plane  of  the  transform  lens. 
With  the  usual  Gaussian  lens  equation,  the  image  of  P2 
can  be  shown  to  be  located  at  a  distance 

L  =  f2/S  (1) 

away  from  input  plane  Pi.  We  now  refer  to  the  far- 
field  (i.e.,  Fraunhofer)  diffraction  arrangement  of  Fig. 
2.  To  achieve  a  Fourier  diffraction  pattern  at  plane 
P2,  it  is  required  that  AL'  be  sufficiently  small  com¬ 
pared  with  the  illuminating  wavelength  i.e., 

A V  <  X/4.  (2) 


Thus  the  allowable  deviation  from  the  focal  length 
should  be 

5  <  2M//6)2.  (6) 

Note  that  this  result  is  known  as  the  focal  tolerance  or 
the  focal  depth  of  the  optical  system.4 

One  of  the  most  important  aspects  of  an  optical 
correlator  must  be  the  shift-invariant  property  of  the 
Fourier  transformation,  from  which  the  position  of  the 
object  (i.e.,  the  target)  can  be  extracted.  We  now 
investigate  the  constraints  in  object  translation  under 
the  quasi-Fourier-transform  regime.  Consider  the 
quasi-joint-Fourier-transform  configuration  of  Fig.  3. 
A  displacement  of  the  object  spectrum  at  the  quasi- 
Fourier  domain  P2  is  anticipated.  Referring  to  the 
triangular  configuration  in  Fig.  3,  the  amount  of  dis¬ 
placement  along  the  vertical  direction  is 

a  =  -fd,  (7) 

where  d  is  the  translation  of  the  object. 

To  ensure  good  overlapping  object  and  reference 
spectra  at  the  quasi-Fourier  domain  P2,  the  amount  of 
displacement  should  be  small  compared  with  the  size 
of  the  main  lobe  of  the  object  spectrum,  i.e., 

a  <  \f/2b.  (8) 


In  view  of  the  similar  right  triangles  of  Fig.  2,  L'  can  be 
approximated  by 


V  ^ 


mf 

2A U  ’ 


(3) 


where  b  is  the  size  of  the  input  object. 

To  obtain  a  Fourier  diffraction  pattern  at  the  obser¬ 
vation  screen,  it  is  required  that  the  inequality  of  rela¬ 
tion  (2)  hold.  Thus  the  distance  to  the  observation 
screen  should  be 

V  >  b2/ 2A.  (4) 

It  is  apparent  that,  to  achieve  a  quasi-Fourier  trans¬ 
formation  at  P2  as  depicted  in  Fq.  ( 1 ) ,  one  should  have 

L  >  U.  (5) 


If  we  simply  substitute  Eq.  (7)  into  relation  (8),  the 
amount  of  allowable  object  displacement  is 


d  < 


fL. 

2b5 


(9) 


Monochromatic 
Plane  Wave 


u — w  6  w- 


Fig.  1.  Quasi-Fourier  transformation.  P,  is  the  image  of 

Pj. 
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Monochromatic 
Plane  Wave  at.’ 


Fig.  2.  Fourier  diffraction  condition. 


object  function 


Fig.  3.  Quasi-joint-Fourier  transformation. 


Beam  Splitter 


Fig.  4.  Nonconventional  joint  Fourier  transformation. 


To  recapitulate,  the  separation  between  the  object  and 
the  reference  functions  should  be  within  the  con¬ 
straint  of  relation  (9).  Under  this  condition  the  object 
and  reference  spectra  will  be  mostly  overlapped,  pro¬ 
ducing  interferometric  fringes  (i.e.,  the  carrier  fre¬ 
quency)  such  that  the  shift-invariant  property  can  be 
preserved. 

We  new  describe  the  nonconventional  joint-trans¬ 
form  teem:'  ue  as  illustrated  in  Fig.  4.  Let  us  assume 
that  2 an  is  the  size  of  the  object  spectrum,  with  am 
written  as 

->m=A/mai=AW/fe,  (10) 


where  fmax  is  the  upper  spatial  frequency  content  of 
the  object  and  W  =  bfmax  is  the  space-bandwidth 
product  of  the  object.  Note  that  the  incident  angle  at 
the  square-law  detector,  formed  by  the  object  and  the 
reference  beams,  can  be  written  as 

6  c*  arctan(5/am),  (11) 

wu  3  6  is  the  maximum  focal  depth  of  relation  (6). 
By  instituting  relation  (6)  and  Eq.  (10)  into  relation 
(11),  the  constraint  of  the  incident  angle  is 

6  <  arctan(2 f/bW),  (12) 

where  f  is  the  focal  length  of  the  transform  lens,  b  is  the 
size  of  the  object,  and  W  is  the  space-bandwidth  prod¬ 
uct  of  the  object.  To  have  an  idea  of  the  magnitudes 
involved,  we  assume  that  the  focal  length  f  =  500  mm, 
the  object  size  b  =  10  mm,  and  the  space-bandwidth 
product  of  the  object  W  =  200;  thus  the  incident  angle 
must  be  6  <  27°. 

We  further  note  that  the  proposed  nonconventional 
joint  Fourier  transformation  may  also  be  achieved 
with  a  different  arrangement,  as  shown  in  Fig.  5,  in 
which  we  assume  that  the  constraints  of  the  focal 


Beam  Splitter 


Fig.  5.  Nonconventional  quasi-joint-Fcu.wr  transforma¬ 
tion. 
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Fig.  6.  (a)  Input  object  and  reference  function,  (b)  Output 
correlation  spots. 
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Fig.  7.  (a)  Input  and  reference  objects,  (b)  Joint-tranS' 
form  holograms. 


6*0  S^f/IO-SOmm  5af/5*100mm 


(a) 


(b) 


Fig.  8.  (a)  Output  correlation  spots,  (b)  Photometer 
traces  of  the  output  correlation  peaks. 

depth  of  relation  (6)  and  the  allowable  incident  angle 
of  relation  (12)  hold.  These  optical  setups  have  the 
advantage  of  producing  a  larger  joint-transform  spec¬ 
trum,  for  which  a  higher  read-out  correlation  peak  can 
be  obtained.  Furthermore,  if  the  target  displacement 
is  within  the  shift-invariant  constraint  of  relation  (9), 
the  proposed  nonconventional  JTC  (NJTC)  can  also 
extract  the  spatial  location  of  the  target. 

To  illustrate  the  shift-invariant  property  of  the 
NJTC,  an  input  transparency  with  four  English  char¬ 
acters  and  a  reference  letter  (Fig.  6(a)]  was  used.  The 
NJTC  structure  used  in  this  experiment  is  shown  in 
Fig.  4.  The  joint-transform  power  spectrum  was  re¬ 
corded  by  a  photographic  transparency,  which  is 
called  a  joint-transform  hologram  (JTH).  By  simply 
illuminating  the  recorded  JTH  with  coherent  light, 
cross  correlations  of  input  characters  with  the  refer¬ 
ence  letter  can  be  viewed  at  the  output  plane  shown  in 


Fig.  6(b).  Two  autocorrelation  peaks,  representing 
the  locations  of  the  input  character  G,  can  be  ob¬ 
served.  Thus  we  see  that  the  shift-invariant  property 
of  the  NJTC  is  preserved.  The  incident  angle  for  this 
experiment  was  set  at  ~10°. 

To  illustrate  the  results  obtained  using  the  quasi- 
Fourier-transformation  arrangement  of  Fig.  5,  the  in¬ 
put  object  and  reference  function  of  Fig.  7(a)  were 
used.  In  this  experiment  three  JTH’s  for  8  =  0, 8  =  f/ 
10  -  50  mm,  and  8- ft 5  =  100  mm  were  recorded  (Fig. 
7(bl].  Throughout  the  making  of  the  set  of  holo¬ 
grams,  the  incident  angle  was  maintained  at  9  as  10°. 
Note  that  the  scale  of  the  joint  power  spectrum  en¬ 
larges  as  <5  increases.  Again,  by  simple  coherent  illu¬ 
mination  of  the  JTH’s,  their  output  correlation  peaks 
were  obtained  [Fig.  8(a)].  Photometer  traces  of  the 
corresponding  correlation  peaks  are  shown  in  Fig. 
8(b).  From  these  results  we  see  that  the  peak  intensi¬ 
ty  increases  as  5  increases,  while  the  size  of  the  correla¬ 
tion  spot  decreases  as  8  increases.  This  phenomenon 
is  primarily  due  to  the  size  of  the  joint-transform  spec¬ 
trum,  as  seen  in  Fig.  7(b).  Thus  the  proposed  NJTC 
has  the  ability  to  improve  the  output  signal-to-noise 
ratio  and  the  accuracy  of  the  signal  detection. 

Furthermore,  with  a  larger  incident  angle  0,  it  is  also 
possible  to  increase  the  spatial  carrier  frequency  of  the 
JTH.  This  would  allow  a  larger  separation  between 
correlation  spots  away  from  the  zeroth-order  diffrac¬ 
tion.  However,  higher  spatial  carrier  fringes  would 
impose  a  higher  resolution  constraint  on  the  square- 
law  detector  used. 

We  have  presented  a  NJTC  technique  that  allevi¬ 
ates  the  inefficient  use  of  the  light  source  as  in  the 
conventional  JTC.  We  have  shown  that  with  a  quasi- 
Fourier  arrangement  the  shift-invariant  property  of  a 
NJTC  can  be  maintained.  The  major  advantages  of 
the  NJTC  are  the  efficient  use  of  incident  light,  the 
use  of  smaller  transform  lenses,  higher  correlation 
peaks,  and  a  higher  carrier  fringe  frequency. 

Since  a  joint-transform  processor5  can,  in  principle, 
process  all  the  information  that  a  conventional  coher¬ 
ent  optical  processor  can,  we  believe  that  the  NJTC 
possesses  many  capabilities  beyond  those  discussed 
here. 

We  acknowledge  the  support  of  the  U.S.  Army  Mis¬ 
sile  Command  through  the  U.S.  Army  Research  Office 
under  contract  DAAL03-87-0147. 
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Abstract.  This  paper  presents  s  technique  of  using  data  association  target 
tracking  in  a  motion  sequence  via  an  aoaptiva  joint  transform  correlator. 
The  massive  dei;  in  the  tie!*  of  view  can  bo  reduced  to  S  few  correlation 
reaks.  The  average  velocity  of  <*  target  during  the  tracking  cycle  is  then 
oc’ennined  from  tne  location  of  the  correlation  peak.  We  have  used  a  data 
association  algorithm  for  the  analysis  of  these  correlation  signals,  with 
which  multiple  targets  can  be  tracked.  A  ohase  mostly  LCTY  is  used  in 
the  hybrid  joint  transform  correlation,  5v»*-»*n,  and  simultaneous  tracking 
of  three  targets  is  demonstrated. 

Subject  terms:  optical  pattern  recognition;  target  tracking,  .,.yuid  crystal  television: 
joint  transform  correlators. 
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1.  INTRODUCTION 

One  of  the  major  advantages  of  optical  signal  processing  is  the 
parallelism  in  handling  high  space-bandwidth  signals.  However, 
the  price  paid  is  the  complexity  and  stringent  alignment  of  an 
optical  system.  On  the  other  hand,  its  digital  counterpart  offers 
flexibility  and  programmability,  while  sacrificing  full  parallel¬ 
ism.  In  this  paper,  we  describe  a  hybrid  optical  technique  to 
tackle  a  multitarget  tracking  problem.  Let  us  assume  that  there 
are  multiple  targets  whose  dynamic  states  (e.:  locations,  ve¬ 
locities,  etc.)  are  to  be  evaluated  in  a  motion  se  nuence.  At  every 
frame  a  new  scene  is  captured  by  an  optical  sensor.  The  space- 
bandwidth  product  in  each  frame  is  so  high  that  it  is  extremely 
difficult  for  an  all  electronic  computer  to  process  on  a  real-time 
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basis.  Although  parallel  processing  can  be  performed  by  an 
optical  system,  it  is  not  feasible  to  build  an  optical  system  that 
performs  precise  decision  making.  The  hybrid  optical  system 
that  we  propose  uses  a  joint  transform  correlator  to  canv  out  the 
conelation  operation  and  a  data  association  algorithm  for  mul¬ 
tiple  target  tracking. 

Target  tracking  with  an  optical  conelator  was  first  demon¬ 
strated  by  VanderLugt  in  1974. 1  Subsequently,  real-time  track¬ 
ing  using  a  VanderLugt  correlator  has  been  reported. :'3  The 
development  of  a  joint  transform  correlator  (JTC),4'5  on  the  other 
hand,  offers  an  alternative  for  target  tracking.6  Simplicity,  easier 
alignment,  and  the  avoidance  of  matched  filter  synthesis  are  the 
major  advantages  of  using  a  JTC.  It  also  has  the  capability  to 
adapt  to  new  input  scenes:  in  the  proposed  architecture,  the 
previous  frame  is  used  as  a  reference  image  to  correlate  with 
the  target  in  the  current  frame.  Assuming  that  the  processing 
cycle  of  the  system  is  sufficiently  short,  then  a  target  will  look 
much  the  same  after  a  few  sequential  frames,  and  hence  strong 
correlation  signals  will  be  generated  at  the  output  plane.  The 
position  measurements  of  the  correlation  peaks  can  be  interpreted 
as  the  aver  velocities  of  the  targets  within  a  tracking  system. 
Based  on  these  measurements,  the  dynamic  states  of  the  targets 
can  be  updated  Cince  multiple  correlation  peaks  are  generated 
in  a  multitarget  backing  problem,  a  data  association  algorithm 
can  be  employed  :c  associate  each  of  the  peaks  to  the  correct 
paths  of  the  targets’  motion.  It  must  be  mentioned  that  the  adap¬ 
tive  property  of  this  system  enables  the  tracking  of  targets  under 
change  in  orientation,  scale,  and  perspective  during  the  course 
of  their  movement. 
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in  this  paper  we  demonstrate  the  use  of  a  hybrid  JTC  based 
on  the  aforementioned  technique  of  multiple  target  tracking.  A 
liquid  crystal  television  tvas  used  in  our  experiment,  and  the 
implementation  of  this  spatial  light  modulator  (SLM)  using  phase- 
mostly  modulation  mode  is  discussed  in  detail.  To  simplify  the 
subject  matter  of  this  pape*-,  the  problem  of  initial  target  acqui¬ 
sition  is  not  discussed  here. 

1.  TRACKING  ALGORITHM 

Let  f'Lr.y),  n  =  I  to  N.  be  the  targets  in  an  image  sequence, 
each  of  which  are  moving  independently.  Two  sequential  scenes 
are  displayed  on  the  input  plane  of  a  JTC  such  that  the  previous 
fra-  te  (k  -  1)  and  the  current  frame  ( k )  are  positioned  in  the 
jppi'  and  lower  halves  of  the  LCTV,  respectively.  The  corre- 
apom.  mg  image  function  is  given  by 

TKvt  - 


:vi:ere  Is-  t>  the  height  of  the  active  aperture  of  the  LCTV, 
Ui  -  i.yi'-  .>  «,*e  the  locations  of  the  targets  in  the  k  -  !  frame, 
and  (b.C.  h'/j.)  «re  the  relative  .ranslatioc  cf  the  targets  from  the 
k  -  1  frant**-  *o  th  :  k  frame.  The  corresponding  joint  transform 
power  spectrum  t  ITP5I  is  then  introduced  as  modulation  at  the 
input  plane  for  the  second  Fourici  transformation,  h  can  be 
shown  that  the  output  complex  light  distribution  is  given  b) 

C(  r.\ )  = 

\  \ 

T  t  _  ,(ar  i  -  a?,  i,  v+ vT- 1 

n  a  1  m  -  | 

N  .V 

-  S  2  RT'tix+xT-  i-hxT-x". 1  +  &x*.  v+jT-i -8>r-vf-i+5yJ) 

n  -  I  m  b  | 

A  V 

*  ^  2  Rt-ixlx+x"- 1  ~xi- 1  +fu*.  y+yP- 1  ->■?- 1  +8>if +o) 

n  -  I  m  ■*  \ 

A  fm 

~  ^jRTi  \(x  +  xT- 1 -hx"- x".  i.  y  +  yf_| -oyT~yl- 1 -a)  .  (2) 

n  s  I  m  =  | 

where 

RTClx-a,  v-p)  -  fT(x.y)  ® /*■(*.>)  ★  8(x--oc.  ;  -pi  (3) 

is  the  correlation  function  between  f?  and /?■  located  at  (a, (3), 
and  (x)  and  ★  denote  the  correlation  and  convolution  operations, 
respectively. 

Note  that  the  autocorrelation  functions  /?";?  and  Rt-\,k-\  are 
located  at  the  origin  of  the  output  plane.  The  correlation  between 
the  same  target  in  the  k  frame  and  the  k  -  I  frame,  given  as. 
R’l "  i,  is  diffracted  around  (Sx",  8vf + a )  and  (-5x*,-8 y"-a). 
The  rest  of  the  terms  ( RTf- 1  and  R™f  for  m  =£  n )  represent  the 
cross-correlation  functions  between  target  m  and  target  n.  It  must 
be  stated  that  in  this  multiple  target  tracking  problem,  we  have 
assumed  that  the  targets  in  any  single  input  frame  do  not  look 
alike,  so  the  correlation  functions  R” f- 1  and  RTf  generate  much 
weaker  correlation  signals  ban  that  gcneraied  by  the  function 
1.  Furthermore,  the  size  and  the  gray  level  of  the  targets 
arc  also  assumed  to  be  relatively  congruous  with  ea-'h  other,  so 
the  intensities  of  the  correlation  peaks  generated  from  Rl't-  i . 
n  =  1  to  N.  arc  not  significantly  different. 


By  translating  the  coordhate  origin  from  the  optical  axis  to 
(Om).  the  correlation  peaks  .hat  are  generated  from  the  same 
target  in  the  k  -  1  an*!  the  k  frame  are  then  located  at 
(8.vi.8y£)  in  this  new  coordinate  system.  If  these  correlation 
peaks  are  associated  with  the  proper  target  motions,  then  the 
locations  of  the  targets  ,n  the  k  frame  are  given  by 

xx  —  a'i - 1  +  8x*,  y!  ~  y*-i  +  8ylt,  n  —  I  to  A’  .  (4) 

Equation  (41  shf'Ws  iiiat  the  new  locations  of  the  targets  in  the 
k  frame  can  be  updated  based  on  their  locations  in  the  previous 
frame  nd  the  position  measurements  of  the  correlation  peaks. 
It  is  apparent  that  the  k  and  k  +  1  frames  can  be  sent  to  the 
input  plane  in  the  next  tracking  cycle,  so  multiple  targets  can 
be  tracked  simultaneously  on  a  near  real-time  basis. 

Note  that  the  position  of  the  correlation  peak  in  the  new 
coordinate  system  represents  the  average  velocity  of  a  target 
during  the  sampling  nterval  St,  i.e., 

x  =  8.r/8 /.  v  =  8v/8f  .  (5) 

Therefore,  with  a  constant  sampling  iterval  and  assuming  that 
the  sensor  is  not  slewing  or  panning,  the  new  coordinate  system 
represents  the  plane  of  average  velocity.  For  example,  targets 
moving  with  constant  velocity  produce  stationary  correlation 
peaks,  targets  moving  with  constant  acceleration  generate  cor¬ 
relation  peaks  that  are  located  at  equally  separated  intervals  on 
a  straight  line,  and  correlation  peaks  located  at  the  origin  of  the 
i.-w  coordinate  plane  correspond  to  stationary  objects  or  back¬ 
ground  scene  In  our  experimental  demonstration,  multiple  tar¬ 
gets  arc  traveling  at  a  different  velocity  and  in  different  direc¬ 
tions.  thus  the  correlation  peaks  generated  are  located  at  different 
positions  at  the  velocity  plane. 

3.  DATA  ASSOCIATION 

In  general,  the  motion  of  the  targets  can  be  represented  by 
dynamic  models,  whicn  are  governed  by  well-known  iws  of 
physics.  Unpredictable  changes  in  target  motions,  commonly 
called  maneuvers,  can  also  be  treated  as  gradual  changes  of 
motion  parameters,  if  the  sampling  interval  is  sufficiently  short 
compared  with  the  maneuver  time.  Therefore,  it  is  not  difficult 
to  associate  the  measurements  in  the  velocity  plane  with  the 
targets,  based  on  the  past  history  of  the  targets.  Any  prior  knowl¬ 
edge  of  the  targets  before  tracking  will  also  be  helpful  in  this 
association  process.  For  example,  if  one  target  is  known  to  travel 
much  faster  than  the  others,  then  its  correlation  peak  is  more 
likely  to  be  loca»ed  farther  away  from  the  origin  in  the  velocity 
plane  than  those  of  slower  ones. 

In  this  section,  a  parameter-based  data  association  algorithm 
using  a  Kalman  filtering  model  is  described.  It  is  assumed  that 
the  sampling  interval  bt  is  sufficiently  short  that  tiie  dynamic 
parameters  (velocity,  acceleration,  angular  velocity,  etc.)  of  the 
targets  arc  fairly  consistent  within  a  few  sequential  frames.  Thus, 
given  the  dynamic  parameters  of  a  target  in  the  k  frame,  the 
parameters  in  the  next  frame  should  be  rather  predictable.  Let 
c(  k )  be  the  measurement  at  the  k  frame  and  :(  k  \  k  -  1 )  be  the 
predicted  measurement  in  the  k  frame,  based  upon  the  infor¬ 
mation  evaluated  up  to  the  k  -  I  frame.  Then,  the  innovation 
(or  measurement  residue),  defined  as 
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v(k)  =  z(k)  -  z(k  i  k-  1)  ,  (6) 

can  be  used  to  evaluate  the  likelihood  of  association  between 
z(k)  and  the  target  under  consideration.  In  stochastic  models, 
one  would  evaluate  the  normalized  squared  distance  D  from  the 
measured  z(k)  to  the  current  track, 

D  =  v'5"'v  ,  (7) 

where  v'  is  the  transpose  of  v  and  S  is  the  innovation  covariance 
of  the  target,  which  is  basically  the  variance  of  the  estimated 
states.  The  values  of  z(k  )  k  -  1),  S,  and  the  like  can  be  eval¬ 
uated  by  applying  a  standard  Kalman  filter  to  the  dynamic  model. 
The  use  of  Kalman  filtr  .ing  in  stochastic  modeling7  is  a  well- 
known  subject  on  its  own  and  will  not  be  discussed  here.  How¬ 
ever,  a  brief  discussion  will  be  given  in  Sec.  7,  followed  by  an 
explanation  of  a  simplified  filtering  process  to  illustrate  the  ca¬ 
pability  of  this  multiple  target  tracking  technique. 

A  data  association  process  can  be  carried  out  as  follows: 

Step  1:  At  the  k  -  1  frame,  the  dynamic  parameters  of  the  N 
targets  are  determined  at  track  I  to  N. 

Step  2:  At  the  k  frame.  N  new  measurements  are  made,  given 
as  a,b . N.  The  normalized  square  distances  are  then  com¬ 

puted: 

D,a  =  v'luSr'v’la.  D,i  =  vifrSr'vift,  CtC.  . 

and  similarly  for  Dia,  Dib . and  so  on. 

Step  3:  The  most  likely  association  is  given  by  choosing  the 
possible  combination  of  Ds  that  yields  the  minimum  sum. 

To  initiate  the  tracker,  all  the  measurements  in  the  first  few 
tracking  cycles  are  used  to  set  up  multiple  potential  tracks.  Tracks 
that  have  inconsistent  dynamic  parameters  are  dropped  until  only 
a  single  track  is  assigned  to  each  target.  Recall  that  the  mea¬ 
surement  in  each  cycle  is  the  average  velocity  of  .*  target  during 
the  sampling  period.  Therefore,  dynamic  parameters  can  be  as¬ 
signed  quickly  in  a  few  cycles:  only  two  or  three  cycles  are 
needed  to  determine  targets  with  constant  velocity  or  constant 
acceleration.  The  tracking  scheme  discussed  so  far.  however, 
does  not  determine  the  initial  positions  of  the  targets  in  the  first 
frame  nor  identify  the  measurements  to  the  targets  due  to  the 
nature  of  this  adaptive  correlation  scheme.  Therefore,  an  initial 
acquisition  scheme  is  needed  to  perform  this  task  before  the  start 
of  the  adaptive  tracker.  This  can  be  done  by  using  prestored 
reference  images  located  at  fixed  positions  at  the  image  plane. 
Further  analysis  of  the  initial  target  acquisition  technique  is  be¬ 
yond  the  scope  of  this  paper. 

4.  DIFFRACTION  EFFICIENCY  AND  RESOLUTION 

In  multiple  object  correlation,  the  JTPS  is  encoded  with  multiple 
carrier  frequencies.  The  detection  efficiency  of  the  system  is 
therefore  inevitably  lower  when  compared  with  that  for  single 
object  correlation.8  Consider  when  N  identical  objects,  each 
located  at  x„.  n  =  1  to  <V  in  a  i-D  space,  arc  the  input  objects 
to  a  JTC.  The  input  function  is  then  given  as 

Tix.v)  =  (8) 

n  \ 

If  the  JTPS  is  linearly  recorded  and  displayed  on  a  SLM.  the 
normalized  amplitude  transmittance  on  (he  SLM  becomes 


,v  ,v 

W+2  2  2cos(x„-.rm)«  . 

I  m>n 

(9) 

where  the  normalization  constant  is  set  to  be  the  magnitude  at 
u  =  0.  The  correlation  output  is  obtained  by  taking  the  inverse 
Fourier  transform  of  and  the  intensity  of  the  correlation 
function  at  x„  -  xm  is  given  as 

C(x,y)  =  jj; *  ^X-Xn+Xm)  I2  .  (10) 

The  l/A'4  factor  seems  to  suggest  that  a  JTC  is  of  little  value  tor 
multiple  object  correlation.  However,  it  is  obvious  that  this  factor 
comes  in  as  the  normalization  constant  in  Eq.  (9).  In  practical 
considerations,  the  energy  at  the  origin  is  much  higher  than  that 
in  the  other  frequency  components;  it  is  therefore  unrealistic  to 
use  this  value  as  the  normalization  constant.  In  fact,  in  most 
cases  the  dynamic  range  of  a  power  spectrum  is  so  large  that 
no  detector  can  record  the  spectrum  linearly.  Therefore,  impos¬ 
ing  an  upper  limit  on  the  power  spectrum  is  unavoidable.  More¬ 
over,  it  can  be  seen  that  the  bracketed  terms  in  Eq.  (9)  are  a 
summation  of  sinusoidal  signals  of  different  spatial  frequencies. 
The  bias  level  build-up  in  this  summation  significantly  reduces 
the  modulation  of  the  fringes.  However,  due  to  the  sensitivity 
of  a  practical  detector,  part  of  this  bias  will  be  eliminated,  hence 
increasing  the  modulation  of  the  fringes.  Therefore,  in  most 
experiments,  replacing  the  normalization  process  on  the  power 
spectrum  with  a  nonlinear  recording  process  bounded  between 
the  iower  and  upper  threshold  is  unavoidable,  and  the  number 
of  objects  in  a  JTC  will  affect  the  diffraction  efficiency  in  a 
highly  nonlinear  fashion.  If  the  upper  and  lower  threshold  are 
properly  chosen  such  that  the  modulation  of  the  fringes  is  am¬ 
plified  by  this  clipping  effect,  then  a  much  better  diffraction 
efficiency  can  be  achieved  even  for  multiple  object  correlation. 
The  thresholding  levels  can  be  adjusted  simply  by  changing  the 
output  power  of  the  illuminating  source,  by  software  program¬ 
ming.  or  by  changing  the  recording  property  of  the  square  law 
detector.9  In  this  experiment,  the  first  two  techniques  mentioned 
were  uti!:_ed  to  optimize  the  correlation  output. 

Due  to  the  nature  of  this  adaptive  tracking  technique,  the 
correlation  peaks  generated  from  different  targets  with  almost 
identical  velocities  will  fuse  together,  hence  reducing  the  res¬ 
olution  of  the  correlation  spots.  If  W  is  the  width  of  a  target, 
then  2W  is  the  minimum  separation  for  the  two  correlation  func¬ 
tions  to  be  completely  separated.  However,  due  to  the  nonlinear 
thresholding  effect  imposed  on  the  power  spectrum,  as  discussed 
previously,  the  width  of  a  correlation  peak  will  be  much  sharper 
than  2 W.  Nevertheless,  the  minimum  separation  for  two  peaks 
to  be  distinguishable  must  be  greater  than  or  i,qual  to  the  full 
width  half  maximum  (FWHM)  of  the  peak.  To  resoKe  the  mea¬ 
surements  for  two  moving  targets,  their  difference  in  velocity- 
must  satisfy  the  condition 

(al/FWHM,)2  +  (Ay/FWHMV):  >  1  .  <H) 

If  the  sampling  interval  is  too  short  compared  with  the  velocities 
of  the  targets,  the  correlation  peaks  will  be  crowded  together 
near  the  origin  (of  the  translated  coordinate)  and  will  become 
unresolvable.  Therefore,  this  resolution  criterion  imposes  a  lim¬ 
itation  to  this  adaptive  tracking  algorithm,  for  which  an  optimum 
sampling  rate  should  be  used. 


I(U'V)  /V2|F(0,0)|-|/r(“’k’)l' 
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|  5.  HYBRID  MODULATION  PROPERTY 

Previous  applications  of  LCTVs  in  optical  processors  mostly 

I  used  the  polarization  modulation  property  of  twisted  nematic 
liquid  crystals.  Amplitude  modulation  can  thus  be  obtained  ac¬ 
cording  to  Malus'/  law.  Even  under  binary  phase-only  modu¬ 
lation,  as  proposed  previously,10,11  the  phase  modulation  ac- 

Itually  results  from  the  polarization  modulation  by  orienting  the 
analyzer  to  be  perpendicular  to  the  bisector  between  the  ON  and 
OFF  states  of  the  LC'TV.  However,  besides  the  polarization 

■  modulation  property  of  a  90°  twisted  nematic  liquid  crystal  cell, 
there  is  also  an  inherent  birefringence  effect  associated  with  the 
molecule.  The  birefringence  is  altered  as  the  structure  of  the 
liquid  crystal  starts  to  deform  when  the  applied  electric  field  is 

I  above  a  certain  threshold  level  (the  twisted  nematic  liquid  crystal 
deformation  threshold).  This  results  in  phase-only  modulation. 
When  the  applied  field  is  sufficiently  hign  and  reaches  another 
threshold  value  (i.e.,  the  optical  threshold),  the  twisted  structure 

I  starts  to  untwist  itself  and  polarization  modulation  takes  place. 
In  this  region,  the  LCTV  is  modulated  in  a  hybrid  mode:  both 
phase  and  polarization  modulations.  Therefore,  phase-mostly 

I  modulation  can  be  achieved  by  setting  the  bias  voltage  of  the 
LCTV  at  the  minimum  value.  This  effect  was  documented  by 
Blinov12  and  has  been  discussed  recently  by  Konforti  et.  al. 
Applications  of  this  phase  modulating  property  have  also  been 

I  reported.14,15 

It  must  be  mentioned  that  for  optimum  phase  modulation 
effect,  the  polarization  direction  of  the  incident  light  must  be 
aligned  parallel  to  the  molecular  director  on  the  LCTV’s  front 

I  surface.  No  phase  modulation  was  observed  in  our  experiment 
when  the  polarization  director  was.  perpendicular  to  the  molecule 
director.  This  has  also  been  mentioned  in  both  Refs.  13  and  14. 

I  This  effect  can  be  explained  as  follows.  When  the  polarization 
plane  of  the  incident  light  is  parallel  to  the  director  of  the  liquid 
crystal  molecules,  the  electric  field  vector  (E-vector)  is  parallel 
to  the  molecules;  hence,  the  incidence  light  becomes  extraor- 

I  dinary  to  the  liquid  crystal  cell.  As  the  electric  field  propagates 
through  the  twisted  cell,  its  plane  of  polarization  follows  the 
twist  such  that  the  E-vector  is  always  parallel  to  the  molecules, 
and  therefore  phase  modulation  is  achieved.  However,  if  the 

I  polarization  plane  of  the  incident  light  is  perpendicular  to  the 
molecule,  then  it  is  an  ordinary  light  to  the  cell.  As  the  electric 
field  propagates  through  the  twisted  cell,  its  plane  of  polarization 

I  remains  perpendicular  to  the  molecule,  and  there  is  no  relative 
phase  shift.  It  is  interesting  to  note  that  the  original  polarizer 
attached  to  the  LCTV  used  in  this  experiment  (Seiko  LVD  202 
color  LCTV,  220  x  240  pixels)  is  found  to  be  aligned  perpen- 

Idicular  to  the  molecular  director.  We  suspect  that  by  doing  so, 
the  undesirable  coloring  effect  due  to  birefringence  can  be  elim¬ 
inated.  and  proper  color  selection  is  solely  controlled  by  internal 
circuitry  driving  the  RGB  pixels. 

I  To  measure  »he  phase  modulation  property  nf  this  LCTV,  we 
have  removed  both  polarizer  and  analyzer  from  ihe  liquid  crystal 
display  panel.  A  Mach-Zehnder  interferometer  similar  to  the  one 

8  mentioned  in  Ref.  13  was  used  to  measure  the  phase  modulation 
property  of  the  LCTV,  The  minimum  bias  voltage  level  (V  = 
0.978.  measured  across  the  brightness  control  switch)  was  set 
at  the  LCTV,  and  the  different  gray  levels  (from  0  to  255) 

I  generated  from  a  Datacube  frame  grabber  (AT428)  were  sent  to 
the  TV  through  a  video/rf  modulator.  The  interference  fringes 
observed  at  the  output  plane  were  recorded,  it  was  found  that 
as  the  gray  level  varied,  the  fringes  started  to  shift,  and  maximum 


Gray  Scale 

Fig.  1.  Plot  of  the  relative  phase  shift  as  a  function  of  gray  level.  The 
gray  level  is  labeled  in  descending  order  since  the  applied  voltage 
increases  as  the  input  gray  level  decreases. 


observed  phase  modulation  was  approximately  340°.  The  phase 
modulation  angle  versus  input  gray  level  is  plotted  in  Fig.  1. 
We  note  that  this  LCTV  is  operated  with  negative  logic:  mini¬ 
mum  input  gray  level  (0  gray  level)  corresponds  to  maximum 
applied  voltage  across  the  cell  and  maximum  gray  level  (255) 
corresponds  to  minimum  applied  voltage,  and  therefore  we  la¬ 
beled  the  x-axis  in  descending  order.  The  polarization  modu¬ 
lation  property  of  the  LCTV  also  was  measured  at  the  same  bias 
voltage,  and  it  was  found  that  polarization  modulation  occurs  at 
gray  levels  below  60.  Therefore,  an  input  image  with  gray  scale 
above  60  is  phase-mostly  modulated,  and  an  image  with  gray 
scale  below  this  value  is  encoded  with  both  phase  <i>  and  polar¬ 
ization  0  modulations. 

To  investigate  the  effect  of  this  hybrid  modulation  in  a  JTC, 
consider  the  plane  of  principal  polarization  to  be  decomposed 
into  two  orthogonal  components  a  and  p.  The  complex  light 
amplitude  behind  the  LCTV  would  be 


£/(*.>’)  =  |£|exp[i<J>(x,>')][dcos0(x,>')  +  |3sin0U.y)l 


where  4>(x.y)  and  0(x,y)  are  the  phase  encoding  and  polarization 
encoding  of  an  input  function  f  \x,y).  If  an  analyzer  is  inserted 
behind  the  LCTV  with  its  director  along  the  d  direction,  then 
only  the  a  component  in  Eq.  (12)  emerges  through  the  LCTV. 
The  image  is  then  subjected  to  phase  and  amplitude  modulations. 
However,  if  the  analyzer  is  not  inserted,  then  both  the  a  and 
3  components  will  contribute  at  the  Fourier  plane.  Since  these 
two  components  are  orthogonal,  the  Fourier  transform  spectra 
resulting  from  the  first  and  second  terms  in  Eq.  (12)  will  add  in 
'erms  of  intensity  rather  then  in  complex  amplitude.  Moreover, 
the  a  and  (3  terms  are  basically  two  reversed  contrast  images: 
'he  power  spectra  of  tin.*"  'wo  images  are  basically  the  same. 
It  can  be  shown  that  if/U.y)  is  a  binary  image,  then  the  two 
power  spectra  are  exactly  identical  except  for  different  dc  mag¬ 
nitudes.  according  to  Babinet’s  principle.  Besides,  it  is  the  fringe 
modulation  that  is  important  in  a  JTC,  and  it  can  be  shown  that 
both  the  a  and  (3  components  produce  the  same  fringe  modu¬ 
lations.  Therefore,  a  JTC  operating  in  hybrid  phase/poiarization 
modulations  ( without  an  analyzer)  would  offer  performance  sim¬ 
ilar  to  a  phase/amplitude  coupled  LCTV'.  The  transmission  of 
the  LCTV,  however,  will  be  much  higher  due  to  the  absence  of 
the  anaivzer. 
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6.  EXPERIMENTAL  SETUP 

Figure  2  shows  the  JTC  architecture  we  have  employed  in  the 
experiment.  The  current  frame  and  the  previous  frame  in  a  mo 
tion  sequence  were  displayed  onto  the  lower  and  upper  halves 
of  the  LCTV.  After  the  first  Fourier  transformation,  the  JTPS 
captured  by  CCD2  was  then  sent  back  to  the  LCTV  for  the 
second  transformation.  The  output  correlation  was  detected  again 
by  CCD2,  with  which  the  locations  of  the  correlation  peaks  were 
determined  by  sequentially  searching  through  the  correlation 
plane  Finally,  a  data  association  algorithm  was  used  to  sort  the 
measurements  and  produced  the  proper  tracks  for  the  targets. 

To  reduce  the  physical  size  of  the  correlator,  we  have  used 
a  negative  lens  behind  the  Fourier  lens  so  that  a  very  long 
effective  focal  length  was  obtained  with  a  relatively  short  phys¬ 
ical  length.16  The  LCTV  also  was  located  behind  the  Fourier 
lens  to  further  reduce  the  size  of  the  JTC.  In  order  to  obtain 
phase-mostly  modulation,  the  bias  voltage  was  set  to  the  lowest 
setting  (0.978  V).  The  input  scene  was  then  binarized  into  two 
particular  gray  levels  to  produce  a  zero  and  ir  phase  difference 
so  that  maximum  modulation  index  on  the  JTPS  was  obtained. 
The  recorded  spectrum  was  not  binarized,  but  the  gray  level  of 
the  fringes  in  the  spectrum  was  reassigned  to  modulate  between 
0°  phase  and  210°  phase  to  maximize  the  diffraction  efficiency.* 
We  stress  that  the  gray  scale  modification  was  operated  without 
the  expense  of  additional  computation  time  by  using  the  iook- 


Fig.  2.  A  compact  joint  transform  correlator  architecture.  The  LCTV 
has  no  polarizer/analyzer  attached.  LI:  collimating  lens;  L2:  Fourier 
transform  lens;  L3:  negative  lens  for  beam  compression.  The  linearly 
polarized  light  was  rotated  to  align  with  the  liquid  crystal  molecule 
director  of  the  LCTV  by  using  a  halfwave  plate. 


up  tables  (the  tables  that  map  the  input  intensity  to  a  gray  scale 
value)  built  into  the  image  frame  grabber.  We  also  note  that  the 
gray  scale  was  modified  to  obtain  maximum  performance  under 
a  phase-only  modulation  assumption,  but  it  may  not  be  the  op¬ 
timum  scheme  if  the  polarization  modulation  effect  is  being 
considered  as  well. 

7.  DEMONSTRATION 

To  demonstrate  the  ability  of  the  tracking  algorithm  that  we  have 
proposed,  a  motion  sequence  of  three  targets  was  generated,  as 
shown  in  Fig.  3.  A  target  tracking  model  can  then  be  set  up  as 
follows:  First,  the  dynamic  state  of  the  target  at  the  k  frame  is 
defined  as 


Since  motion  along  each  coordinate  can  be  decoupled,  only  one 
direction  needs  to  be  considered.  Acceleration  and  maneuvering 
of  targets  can  be  handled  by  including  processing  noise  n  in  the 
dynamic  equation: 


and  the  measurement  is 

i(k)  =  (0  l)x(jt)  +  w  .  (15) 

where  tv  is  the  measurement  noise.  A  Kalman  filtering  process 
can  therefore  be  applied  to  the  tracking  problem.  The  innovation 
covariance  5  can  be  computed,  so  the  normalized  squared  dis¬ 
tance  D  in  Eq.  (7)  can  be  evaluated  and  the  data  association 
process  described  in  Sec.  3  can  be  carried  out.  However,  our 
goal  here  is  to  demonstrate  the  capability  of  the  optical  tracker 
and  to  illustrate  the  data  association  algorithm.  Therefore,  in 


•For  a  sinusoidal  phase  grating  expficbcosHvc)  (|<|>|  =  maximum  modulation 
angle),  the  diffraction  amplitude  m  the  first  order  is  proportional  to  the  Bessel 
function  ./i(<i>).  wherc/i(<h)  >s  maximum  at  105°.  By  considering  the  fringe 
modulation  in  a  JTPS  as  a  sinusoidal  grating,  maximum  diffraction  efficiency 
is  obtained  by  assigning  the  fringes  to  modulate  between  0°  and  210° 


Fig.  3.  Motion  sequence  of  three  targets:  helicopter  and  jet,  moving  from  bottom  to  top; 
airplane,  stationary. 
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Fig  4.  (a)  The  image  sequence  as  displayed  on  the  LCTV  at  each  step  of  the  tracking  cycle, 
(b)  The  correlation  results  as  captured  by  the  CCD  camera  at  the  conesponding  tracking  cycle. 


Fig.  5.  A  3-D  plot  of  the  correlation  result  obtained  in  one  of  the 
tracking  cycles.  Height  of  peaks  are  linear  in  intensity. 

order  lo  show  the  basic  performance  rather  than  the  tillered 
results  in  this  demonstration,  we  simply  assume  a  noise-tree 
measurement  situation  t  >t  =  0).  This  immediately  sets  the  earn 
of  the  (titer  to  one  The  update  dynamic  states  m  the  target  arc 
then  solely  determined  by  the  new  measurements,  and  the  pre¬ 
dicted  measurement  is  given  by 

h  k  -  1 1  -  •  1  -  1 1  ■  ifn 


For  simplicity,  the  value  of  5  is  also  assigned  as  the  variance 
of  the  target's  velocity  in  a  number  of  tracking  cycles. 

A  computer  program  was  written  that  controls  the  joint  trans¬ 
form  process  a.;  well  as  the  data  association  process  based  on 
the  above  simplification  and  the  discussion  in  Sec.  3.  To  initiate 
the  coordinate  the  system  was  first  set  to  run  with  a  stationary 
object  as  input,  and  .he  position  of  the  correlation  peak  thus 
generated  was  assigned  as  the  origin  of  the  velocity  piane  co¬ 
ordinate.  The  tracking  program  was  then  set  to  run  autonomously 
with  the  motion  sequence  in  Fig.  4  as  inputs.  The  computer 
program  was  written  in  Microsoft  C.  which  ran  on  a  microcom¬ 
puter  using  an  INTEL  586  processor  operated  at  25  MHz.  and 
the  processing  time  for  tracking  cycle  was  approximately  0.7  s. 
Figures  4(a)  and  4(b)  ds  pict  the  images  as  displayed  on  the  LCTV 
and  the  corresponding  correlation  spots  generated  during  each 
step  of  the  tracking  cycle.  A  3-D  plot  of  correlation  peaks  is 
also  given,  in  Fig.  5  To  avoid  unnecessary  data  transfer,  the 
current  frame  was  alternately  lo  wed  onto  the  upper/ lower  halt 
ol  the  LCTV  at  every  tracking  cycle  This  modification  generate . 
a  correlation  plane  whose  coordinate  axes  are  inserted  at  each 
cycle,  and  the  location  of  the  measurements  have  to  be  inter¬ 
preted  accordingly  Table  I  s-.rnmanzes  the  empirical  data,  and 
the  (inai  resuits  are  shown  in  Fig  6.  in  which  ihe  actual  locations 
ol  the  targets  at  each  step  ol  the  tracking  cycle  are  also  provided 
for  comparison  ll  can  he  seen  mat  a  relatively  simple  hybrid 
optical  architecture  is  capable  ol  tracking  multiple  largets  rather 
accurately 

8.  CONCLUSION 

We  have  demonstrated  a  data  association  inultttargct  tracking 
technique  using  a  hybrid  JTC  The  role  *>t  the  correlator  is  to 
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TABLE  I.  Empirical  data  after  11  tracking  cycles. 
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Fig.  6.  Final  results  obtained  after  1 1  tracking  cycles.  The  actual  locations  of  the  targets  at  each 
of  the  tracking  cycles  are  also  given  for  comparison.  The  initial  locations  of  the  paths  were 
assigned  at  the  correct  positions  in  this  figure  through  manual  interaction. 


reduce  the  information  of  an  input  scene  with  large  space-band¬ 
width  product  into  a  few  correlation  signals  (velocity  measure¬ 
ments)  so  that  a  digital  system  is  capable  of  tracking  multiple 
targets  at  real-time.  Simplicity  in  optical  architecture,  relaxation 
of  alignment  requirements,  and  the  adaptive  property  arc  the 
major  advantages  of  a  hybrid  JTC  in  this  mode  of  operation. 
The  phase  modulation  property  of  a  LCTV  has  been  investigated 
and  was  employed  in  this  experiment  to  improve  the  detection 
efficiency  of  a  JTC.  A  motion  sequence  of  three  targets  was 
used  as  the  input  scenes  to  demonstrate  the  applicability  of  this 
technique,  and  satisfactory  results  were  obtained. 
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Abstract.  One  of  the  features  in  neural  computing  must  be  the  ability  to 
adapt  to  a  changeable  environment  and  to  recognize  unknown  objects. 
This  paper  deals  with  an  adaptive  optical  neural  network  using  Kohonen's 
self-organizing  feature  Tiap  algorithm  for  unsupervised  .  arning.  A  com¬ 
pact  optical  neural  network  of  64  neurons  using  liquid  crystal  televisions 
is  used  for  this  study.  To  test  the  performance  of  the  self-organizing  neural 
network,  experimental  demonstrations  and  computer  simulations  are  pro¬ 
vided.  Effects  due  to  unsupervised  teaming  parameters  are  analyzed.  We 
show  that  tbs  optical  neural  network  is  capable  of  performing  both  un¬ 
supervised  teaming  and  pattern  recognition  operations  simultaneously, 
by  setting  two  matching  scores  in  the  learning  algorithm.  By  using  a  slower 
learning  rate,  the  construction  of  the  memory  matrix  becomes  more  or¬ 
ganized  topologically.  Moreover,  the  introduction  of  forbidden  regions  in 
the  memory  space  enables  the  neural  network  to  team  new  patterns  with¬ 
out  erasing  the  old  ones. 


i 

i 

i 

I 

l 

i 

l 

t 

i 

I 

ft 


Subject  terms:  optical  pattern  recognition:  self -organizing  neural  networks:  un¬ 
supervised  learning:  Kohonen’s  feature  map:  optical  neural  networks 
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1.  INTRODUCTION 

II  is  well  known  that  digital  computers  can  soh  ;  computational 
problems  millions  of  times  faster  than  human  brains.  However, 
for  cognitive  tasks,  such  as  recognition,  teaming,  a nd  adaptation 
in  a  complex  environment,  the  performance  of  the  human  brain 
is  beyond  the  reach  of  electronic  computers  The  human  brain 
consists  of  millions  of  neurons,  with  each  neuron  interconnected 
to  thousands  of  other  neurons.  Artificial  neural  networks  { ANNs) 
have  adopted  this  unique  feature  of  massive  interconnections . 
simulating  the  learning  capability  of  a  biotogicai  system.  "  ’ 
Strictly  speaking,  two  types  of  learning  processes  are  used 
in  the  human  brain:  supervised  and  unsupervised  learning.'  If  a 
teacher  first  organizes  the  information  and  then  reaches  the  stu¬ 
dents.  the  learning  process  is  obviously  supervised.  For  example, 
in  an  artificial  neural  network.  boih  the  input  and  desired  output 
data  must  be  provided  as  training  examples.  In  otter  wards,  the 
ANN  has  to  be  taught  when  to  fcam  ami  when  to  process  ite 
information.  Nevertheless,  if  an  unknown  object  is  presented  to 
ite  artificial  neural  network  during  the  processing,  the  network 
may  provide  an  erroneous  output  result. 
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On  the  other  hand,  for  unsupervised  learning  (also  called  self- 
learning).  students  learn  by  themselves,  relying  on  some 
simple  rules  and  their  past  experiences.  In  an  ANN.  only  the 
input  data  are  provided,  while  the  desired  output  result  is  not. 
After  a  single  trial  or  series  of  (rials,  an  evaluation  role  (pre¬ 
viously  provided  to  the  neural  network),  is  used  to  evaluate  the 
performance  of  the  network.  Thus,  we  see  that  the  network  is 
capable  of  adapting  and  categorizing  unknown  objects.  This  kind 
of  seif-organizing  process  is  a  representation  of  the  setf-leaminc 
ability  of  the  human  brain. 

In  past  decades,  models  of  ANNs  have  been  developed.  To 
rune  a  few.  the  penrepenxi.  the  error-driven  backpropagadon. 
the  HopfieM  models,  end  the  Boltzmann  machine  are  examples 
of  the  supervised  learning  models.  The  adaptive  resonance  the 
orv  (ART)/  the  oeocogmtroo.5  the  madline.  and  the  Kobonen 
seif-organizing  feature  reap*  are  among  the  best  known  unsu- 
perused  learning  models.  Among  these  models,  the  Kobonen 
model  is  the  simplest  seif-organizing  algorithm,  capable  of  per¬ 
forming  statistical  pattern  recognition  and  classification,  and  it 
can  be  modified  for  optical  neural  re; work  implementation. 
However.  Kohonen's  model  was  originally  designed  for  feature 
mapping  of  statistical  data:  to  perform  adaptive  pattern  recog¬ 
nition.  we  define  three  matching  parameters  and  incorporate 
matching  criteria  in  the  learning  algorithm.  The  effects  of  to 
ami  slow  teaming  are  analyzed.  The  concept  of  ferotdden  regions 
in  the  memory  space  is  introduced  m  the  Kobonen  model  such 
that  ite  neural  network  learns  new  input  objects  without  erasing 
ok!  memory 

The  development  of  ANNs  has  recently  stimulated  great  in¬ 
terest  for  optical  implementations.  ,c‘  ‘®  Because  of  large  infor¬ 
mation  capacity  and  the  eapabUtty  of  establishing  threc-dimemiooai 
massive  intefconnections.  optics  is  considered  a  potential  can¬ 
didate  for  constructing  massively  parallel  neural  networks.  Tire 
first  optical  neural  network,  based  on  a  vector- mat m  processor. 
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Output  space:  MxM  output  neurons 


Input  space:  NxN  input  neurons 


Fig.  1  Single  layer  neural  network. 


Memory  space: 


Fig.  2.  Memory  vectors  in  the  memory 
matrix  space. 
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Fig.  3.  Neighborhood  selection  in  the  mem¬ 
ory  matrix  space  using  Kohonen's  self¬ 
learning  algorithm:  the  initial  neighboring 
region  is  large.  As  time  proceeds,  the  region 
shrinks  until  it  reduces  to  one  memory  sub¬ 
matrix. 


was  proposed  by  Farhat  and  Psaltis  and  their  coworkers  in  1985. 10 
The  optical  neural  network  is  composed  of  a  lenslet  array  to 
establish  the  optical  interconnections  between  the  input  light 
emitting  diode  (LED)  array  and  the  interconnection  weight  ma¬ 
trix  (IWM).  However,  the  architecture  required  a  high  resolution 
spatial  light  modulator  (SLM).  which  is  expensive  to  build  and 
may  not  be  available  in  the  near  future.  To  alleviate  these  short¬ 
comings,  we  developed  an  optical  neural  network  using  an  in- 
exptr  we  video  monitor  for  displaying  a  programmable  rWM. 1415 
Recw.dy,  a  compact  optical  neural  network  using  pocket  size 
liquid  crystal  televisions  (LCTVs)  has  also  been  built.16  We 
shall  use  Kohon*  ’s  self-organizing  algorithm  for  the  unsuper¬ 
vised  learning  in  ,ne  compact  optical  architecture. 

2.  KOHONEN’S  SELF-ORGANIZING  FEATURE  MAP 
ALGORITHM 

Knowledge  representation  in  human  brains  is  generally  at  dif¬ 
ferent  levels  of  abstraction  and  assumes  the  form  of  a  feature 
map  in  the  real  world.  The  Kohonen  model  suggests  a  simple 
learning  rule  by  adjusting  the  interconnection  weights  between 
input  and  output  neurons  based  on  the  matching  score  between 
the  input  and  the  memory.  A  single  layer  neural  network  is 
defined  that  consists  of  NxN  input  and  MxM  output  neurons 
forming  an  input  and  output  vector  space,  as  illustrated  in  Fig. 

1 .  Notice  that  the  output  neurons  are  extensively  interconnected 
to  the  input  neurons  with  weighted  interconnections.  The  output 
nodes  are  also  laterally  interconnected  with  each  other.  Let  us 
assume  that  2-D  vectors  (i.e..  input  patterns)  are  sequentially 
presented  to  the  neural  network,  given  by 

x(f)  =  {.r„{r)  .  ij  -  1.2,.,..  ,V}  .  (I) 

where  t  represents  the  time  index,  such  as  the  iteration  number 
in  discrete-tirr  e  sequence,  and  (ij)  specifies  the  position  of  the 
input  neuron.  Thus,  the  output  vectors  of  the  neural  network 
can  be  expressed  as  the  weighted  sum  of  the  input  vectors 

N  N 

>•«(/)  =  2  2  ««,/ f  >*„(/)  •  L*  =  1,2,...,  M  .  (2) 

<m  ■/-  • 

where  v,*(r)  represents  the  state  of  the  (l.k) th  neuron  in  the 
output  space  and  mih/ft)  is  the  interconnection  weight  between 
the  (ij) th  input  and  the  (l,k) th  output  neurons.  Equation  (2)  can 
also  be  written  in  matrix  innerproduct  representation  given  by 


yttU)  =  m n(t)x(t)  ,  (3) 

where  m«(r)  can  be  considered  to  be  a  two-dimensional  vector 
in  a  4-D  memory  matrix  space,  which  can  be  expanded  in  an 
array  of  2-D  submatrices,  as  shown  in  Fig.  2.  Each  submatnx 
can  be  written  in  the  form 


miui(l)  m«|2(0 

■  fUlklNiO 

mmi(t)  "limit)  ., 

■  W/t2/dri 

_  miwi(t)  m/unU)  ■ 

m  K/wU)  _ 

where  the  elements  in  each  submatrix  represent  the  associative 
weight  factors  from  each  of  the  input  neurons  to  one  output 
neuron. 

Notice  that  the  Kohonen  model  does  not  in  general  specify 
the  desired  output  results.  Instead,  a  similarity  matching  criterion 
is  defined  to  find  the  best  match  between  the  input  vector  and 
the  memory  vectors  and  then  determine  the  best  matching  output 
node.  The  optimum  matching  score  dc  as  defined  by  Kohonen 
can  be  written  as 

dc(t)  -  ||x(r)  -  mr(/)||  (5) 

=  min{d/i(/)}  =  inin{!|x(r)  -  m«(Oll}  ,  (6) 

l.k  l.k 

where  c  —  (l.k)*  represents  the  node  in  the  output  vector  space 
at  which  a  best  match  occurred.  ||j|  denotes  the  Euclidean  dis¬ 
tance  operator. 

After  obtaining  an  optimum  i  latching  position  c,  a  neigh¬ 
borhood  Nc(t)  around  the  node  c  is  further  defined  for  IWM 
modification,  as  shown  in  Fig.  3.  Notice  that  the  memory  matrix 
space  is  equivalent  to  the  output  space,  in  which  each  submatrix 
corresponds  to  an  output  node.  As  time  progresses,  the  neigh¬ 
borhood  Nc(t)  will  slowly  reduce  to  the  neighborhood  that  con¬ 
sists  of  only  the  selected  memory  vector  mc,  as  illustrated  in 
the  figure. 

Furthermore,  a  simple  algorithm  is  used  to  update  the  weight 
factors  in  the  neighborhood  topology,  by  which  the  similarity 
between  the  stored  memory  matrix  m/x(r)  and  the  input  vector 
x(r)  increases.  Notice  that  the  input  vectors  can  be  either  binary 
or  analog  patterns,  while  the  memory  vectors  mv  updated  in  an 
analog  incremental  process.  The  adaptation  formula  of  the  al¬ 
gorithm  can  be  written  as 


1108/  OPTICAL  ENGINEERING  /  September  1990  /  Vol.  29  No.  9 


SELF-ORGANIZING  OPTICAL  NEURAL  NETWORK  FOR  UNSUPERVISED  LEARNING 


134 


LCTV1 

Incoherent  & 

Light  Source  Diffuser  Lenslet  Array  LCTV2 


Imaging  OCD 
Lens  Array 


Fig.  4.  Schematic  diagram  of  a  seif-organizing  optical  neural  net¬ 
work. 


Fig.  5.  Experimental  setup  of  a  self-organizing  optical  neural  net¬ 
work. 


m/*(0  +  ait)[x(t)  -  m,t(0]  . 

for  l.kGNAD  . 

M  M 

m  dt+  1)  =  ■ 

.  m ,*(/)  . 

,  O) 

otherwise  . 

y«(/+ 1)  =/ 

yikU)  -  e  2  2  .WO 

mm  1  n  —  1 

where  0  <  a(t)  <  1  represents  a  gain  sequence,  called  the 
learning  speed,  that  is  usually  a  monotonically  slowly  decreasing 
function  of  t.  In  practice,  the  learning  speed  is  assumed  to  be 
linear,  which  can  be  written  as 

a(t)  =  a(0)  -  at  .  (8) 

where  a  is  a  learning  rate. 

Mention  must  be  made  that  the  point  density  of  the  memory 
vectors  tends  to  approximate  the  probability  density  function  of 
the  input  vectors,  which  has  been  proved  by  Kohonen.6 

It  should  be  noted  that  Eq.  (6)  can  be  expanded  in  the  fol¬ 
lowing  form: 

N  ,V 

dik  -  22  l-MO  “ 

I”  l 

/V  /V  ,V  V  s  N 

=  224<0  +  22 -  2  22m, ^t)  ,  (9) 

i -  ly=  I  i*  I/*  I  i“l/=l 


where  /  is  a  nonlinear  operator  over  the  output  nodes  and  e  is 
an  inhibition  constant  between  the  output  nodes.  For  simplicity, 
we  assume  that  the  nonlinear  operation  f(x)  is  a  thresholding 
function,  given  by 

f  I  ,  xma  , 

f(x)  =  |  x  .  b*zx<a  ,  (12) 

[  -1  ,  x<b  , 

where  a  and  b  are  arbitrary  constants.  By  referring  to  the  MAX- 
NET  algorithm,  we  see  that  each  node  will  excite  itself  and 
inhibit  the  other  nodes  during  each  iteration.  Finally,  only  the 
maximum  output  node  will  retain,  while  all  of  the  other  nodes 
will  eventually  go  to  zero.  Since  the  MAXNET  algorithm  will 
always  converge,  for  e  <  MM,  the  maximum  node  can  always 
be  found.2 


3.  LCTV  OPTICAL  NEURAL  NETWORK 


where  the  first  term  is  a  constant  with  regard  to  the  output 
position  (l,k).  If  the  weight  vectors  are  normalized  so  that  their 
autocorrelations  (i.e. .  the  sum  of  the  squared  weights  from  all 
inputs  to  each  output  node)  are  identical,  then  the  second  term 
also  becomes  a  constant.  Under  this  condition,  the  minimum 
Euclidean  distance  occurs  whenever  the  third  term  becomes  max¬ 
imum,  i.e.. 


[  -v  »  1 

de(t)  =  mm{4(i))  =  max-!  2  2 /)*,,(/) 


l 


=  i/=  i 


(10) 


which  represents  the  output  result  of  the  neural  network.  In  other 
words,  selecting  the  minimum  Euclidean  distance  between  the 
memory  vectors  and  the  input  pattern  vector  is  equivalent  to 
finding  the  maximum  output  node  in  the  output  space. 

The  maximum  output  node  can  be  determined  by  using  ex¬ 
tensive  lateral  inhibition  among  the  output  nodes  in  the  ojtput 
space,  which  is  known  as  the  MAXNET  algorithm.2  The  expres¬ 
sion  of  the  inhibit  interconnections  can  be  written  as 


Recent  developments  in  liquid  crystal  display  technology  have 
made  the  imaging  quality  approach  that  of  commercially  avail¬ 
able  video  monitors.  For  instance,  the  contrast  ratio  of  a  recent 
LCTV,  with  built-in  thin-film  transistors  (TFTs),  is  measured  at 
about  15:1  under  white  light  illumination  (it  can  be  higher,  than 
30:1  if  illuminated  by  laser  light17);  the  dynamic  range  is  about 
16  gray  levels,  which  can  be  continuously  adjusted;  and  the 
transmittance  of  the  LCTVs  is  in  the  range  of  5%  to  10%.  Thus, 
the  LCTVs  can  be  used  to  build  inexpensive  optical  neural  net¬ 
works. 

The  schematic  diagram  of  an  LCTV  optical  neural  network 
and  the  experimental  setup  are  shown  in  Figs.  4  and  5.  A  neural 
network  of  8  x  8  =  64  interconnected  neurons  was  recently 
constiucted16  for  which  an  80  W  xenon  arc  lamp  was  used  as 
the  incoherent  light  source.  In  this  architecture,  a  Hitachi  LCTV 
(LCTV1)  is  used  for  the  generation  of  the  IWM.  which  consists 
of  an  8  x  8  array  of  submatrices,  and  each  submatnx  has  8  x  8 
elements.  This  IWM  is  then  displayed  on  a  fine  diffuser  im¬ 
mediately  behind  LCTV1.  A  Seiko  LCTV  (LCTV2)  is  used  as 
an  input  device  for  the  generation  of  the  input  patterns.  The 
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Fig.  6.  Experimental  demonstration  of  a  self-organizing  optical  neural  network  using  unsu¬ 
pervised  learning:  (a)  Input  patterns,  (b)  Initial  memory  matrix  space  with  random  noise.  In  Final 
memory  matrix  space.  The  four  patterns  are  adapted  in  the  memory  matrix  space;  the  tree  is 
centered  at  (1,8),  the  dog  at  (7,1),  the  house  at  (7,7),  and  the  airplane  at  (1,1). 


lenslet  array,  which  consists  of  8  x  8  lenses,  provides  the  inter¬ 
connections  between  the  IWM  and  the  input  pattern.  Each  lens 
of  the  lenslet  array  images  each  of  the  IWM  submatrices  onto 
the  input  I.CTV2  to  establish  the  proper  interconnections.  Thus, 
the  IWM  submatrices  can  be  superimposed  to  illuminate  the 
input  vector  pattern  of  LCTV2.  This  represents  the  matrix-vector 
product  operation  [i.e.,  the  interconnective  part  of  Eq.  (10)]. 
The  output  result  from  LCTV2  is  then  picked  up  by  a  CCD 
array  detector  for  MAXNET  operation.  The  final  result  can  be 
obtained  by  a  microcomputer.  Notice  that  the  data  flow  in  the 
optical  system  is  controlled  primarily  by  the  computer.  For  in¬ 
stance,  the  IWMs  and  the  input  patterns  can  be  written  onto 
LCTV 1  and  LCTV2,  and  the  computer  can  also  make  decisions 
based  on  the  output  results  of  the  neural  network.  Thus,  the 
proposed  LCTV  optical  neural  network  is  a  programmable  and 
adaptive  neural  network. 


4.  UNSUPERVISFn  LEARNING  IN  THE  OPTICAL 
NEURAL  NET”’  K 

Before  apply'  ,_lf-organizing  model  in  the  optical  neural 
network,  tw*.  ,  <■  Actors  must  be  considered:  one  is  the  system 
components  al,  nent  errors,  the  other  is  the  effect  of  the 

parameters  of  the  sei.  organizing  system. 

Since  the  IWM  submatrices  are  precisely  interconnected  with 
the  input  pattern  vector  by  a  lenslet  array,  the  alignment  of  the 
interconnections  is  rather  critical.  Although  the  adjustment  of 
lenses  is  formidable,  it  is  rather  simple  to  shift  the  IWM  sub¬ 
matrices  in  LCTV1.  A  set  of  test  patterns  can  be  displayed  on 
LCTV1  and  LCTV2.  The  test  patterns  are  shifted  in  small  steps 
according  to  the  sharpness  of  the  output  pattern  detected  by  the 
CCD  camera.  Thus,  the  interconnection  alignment  can  be  self- 
adjusted  via  the  feedback  loop. 

The  uniformity  and  stability  of  the  light  illumination  also 
pose  some  problems  on  the  accuracy  of  the  output  results.  To 
alleviate  the  uniformity  problem,  a  test  is  performed  when  a 
new  input  pattern  vector  is  presented.  During  the  test,  all  ele¬ 
ments  in  the  memory  matrix  are  set  to  the  same  value,  i.e., 

mmjUo)  =  i  .  U.  =  1.2...,,  M  .  i.j.  =  1,2 .  N  .  (13) 


Then  an  output  array  can  be  obtained,  such  as 

.v  ,v 

v’/tUo)  =  22  Mo)|  .  U  -  1.2 . M  .  (14) 

I  *=  \j»  I 

The  uniformity  test  array  Y(/o)  is  then  divided  by  the  output 
array  during  the  self-learning  and  recognition  processes,  given 
by 

y’dt)  =  t~  ,  l.k  -1,2 . M  ,  (15) 

ytkUo) 

where  Y*(r)  is  the  normalized  output  array  in  which  the  non¬ 
uniformity  for  the  light  illumination  can  be  eliminated. 

Another  problem  with  the  light  source  is  the  fluctuation  of 
the  arc  lamp,  which  causes  detection  errors.  To  overcome  this 
problem,  we  took  a  sequence  of  output  frames  and  performed 
average  operations  for  every  iteration. 

For  experimental  demonstrations,  four  8x8  pixel  binary  in¬ 
put  patterns  (i.e.,  a  tree,  a  dog,  a  house,  and  an  airplane)  were 
sequentially  presented  at  the  input  LCTV2  shown  in  Fig.  6(a). 
Figure  6(b)  shows  the  initial  memory  matrix  (i.e.,  IWM)  as  a 
random  pattern.  The  first  neighboring  region  is  chosen  as  Nc( 0) 
=  5,  the  initial  learning  speed  is  a(0)  =  0.01,  and  the  learning 
rate  is  selected  as  in  computer  simulations,  which  is  a  =  0.000025 
per  iteration.  The  output  pattern  picked  up  by  the  CCD  camera 
must  be  normalized  by  referring  to  Eq.  (15).  Then  the  location 
of  the  maximum  output  intensity  can  be  identified  by  using  the 
MAXNET  algorithm.  The  memory  submatrices  in  the  neigh¬ 
borhood  of  the  maximum  output  spot  are  adjusted  based  on  the 
adaptation  rule  of  the  Kohonon  model.  The  amount  of  analog 
increase  in  the  memory  vectors  can  be  controlled  by  the  learning 
rate  a.  The  updated  memory  matrix  is  then  displayed  on  LCTV1 
for  the  next  iteration,  and  so  on.  Due  to  the  gray  level  limitation 
in  LCTV  displays,  a  few  iterations  in  the  learning  process  may 
result  in  one  noticeable  gray  level  change  by  the  CCD  detector 
anay .  Since  the  inputs  are  binary  patterns  at  this  stage,  the  center 
memory  vector  in  the  neighborhood  region  eventually  converges 
to  the  binary  input  vector.  Thus,  the  limited  dynamic  range  of 
LCTVs  would  not  pose  a  major  problem  in  our  experiment.  As 
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Fig.  7.  Feature  map  in  memory  matrix  space.  Three  similar  patterns 
are  located  on  the  same  side  in  the  memory  matrix,  B  centered  at 
(8,4),  C  at  (6,1),  and  0  at  (8,8),  while  pattern  A  sits  in  the  opposite 
direction  of  the  memory  matrix,  centered  at  (14). 


shown  in  Fig.  6(c),  after  400  iterations  the  memory  has  learned 
the  four  input  patterns,  mapping  them  around  four  overlapping 
regions  in  the  IWM.  The  centers  of  these  patterns  are  located 
at  (1,8),  (7,1),  (7,7),  and  (1,1),  respectively. 

One  of  the  major  advantages  in  adaptive  pattern  recognition 
is  to  learn  the  unknowns  in  addition  to  recognizing  known  ob¬ 
jects.  This  task  can  be  achieved  by  adding  three  criteria  to  the 
Kohonen  learning  rule,  for  which  the  matching  rate  R  for  pattern 
recognition  can  be  defined  as 


N  N 


R  = 


I1K,(0|| 


2  2 

_  l»l)°l _ 


iV  rV 

22*5(0 

i-i/”  i 


06) 


where  ||  j|  is  the  norm  operator.  Let  us  assume  that  two  constants 
F]  and  Fi  represent  the  matching  and  unmatching  scores,  by 
which  F\  <  1 ,  Fz  <  1 ,  and  F\  >  Fz ;  then  the  criteria  of  the 
learning  rules  are  as  follows:  (a)  If  R  >  F\,  the  input  object 
matches  the  memory  submatrix  at  position  c.  Then  the  neural 
network  recognizes  the  input  object,  (b)  If  Fz  ^  R  <  F\ ,  the 
input  object  is  identified  as  a  pattern  that  belongs  to  the  class  of 
patterns  (i.e.,  submatrices)  around  the  location  c.  This  shows 
that  the  input  object  has  some  differences  from  the  existing 
patterns  in  the  class.  Then  the  memory  submatrices  in  the  neigh¬ 
borhood  of  c  should  be  modified  to  adapt  to  the  new  input  pattern, 
(c)  If  R  <  Fz,  the  input  object  is  considered  to  be  an  unknown 
pattern.  Then  a  new  category  is  developed  to  learn  the  unknown 
pattern  using  Eq.  (7). 

For  illustration,  F\  and  Fz  are  selected  as  Fj  =  95%  and  Fz 
=  75%.  A  computer-simulated  self-learning  process  is  illus¬ 
trated  in  Fig.  7.  in  which  four  standardized  English  letters  A, 
B,  C.  and  D  are  sequentially  presented  as  the  input  objects  to 
the  neural  network.  It  can  be  seen  that  in  the  memory  matrix, 
the  locations  of  the  submatrices  reflect  the  similarity  among  input 
patterns.  The  grouping  of  the  submatrix  locations  is  known  as 
Kohonen’s  feature  map,  for  which  the  patterns  of  similar  features 
tend  to  stay  close  to  each  other.  For  example.  B,  C,  and  D  are 
similar  patterns;  their  submatrices  are  located  on  the  same  side 
of  the  memory  matrix,  centered  at  (8.4),  (6.1).  and  (8.8),  re¬ 
spectively,  while  pattern  A  sits  in  the  opposite  direction,  located 
at  (1,8). 


It  should  be  noted  that  the  innerproducts  of  the  4-D  memory 
matrix  and  the  2-D  input  pattern  are  performed  in  parallel  by 
the  optics,  while  the  MAXNET  algorithm,  the  thresholding,  and 
the  adaptive  operations  are  performed  by  a  microcomputer. 

Our  experiences  have  shown  that  the  best  results  in  self¬ 
organization  are  obtained  when  the  neighborhood  of  the  memory 
matrix  space  is  first  selected  as  a  wide  region  and  then  allowed 
to  shrink  as  time  proceeds.  The  linearly  decreasing  learning 
speed  a(t)  is  illustrated  in  Fig.  8.  Since  ai  >  a?*  it  is  clear  that 
ai(t)  is  a  faster  learning  speed  than  az(t).  It  is  apparent  that  to 
reach  the  stable  state  of  the  memory  matrix,  ai(/)  would  require 
fewer  iterations  than  az(t).  However,  in  real  life,  the  intensive 
training  may  not  always  be  effective.  With  the  slower  learning 
speed,  the  memory  matrix  is  more  organized  and  also  appears 
smoother.  The  price  paid  is  obviously  the  training  time  required 
for  using  the  slower  learning  speed.  As  an  example,  the  memory 
matrix  space  adapted  from  the  faster  learning  speed  is  shown  in 
Fig.  9(a).  After  400  iterations,  using  a(0)  =  0.02  and  a  = 
0.00005,  the  similar  patterns  I,  J,  and  T  are  adapted  in  the  lower 
left  part  of  the  memory  matrix,  centered  at  (5,1),  (7,2),  and 
(8,4),  respectively.  The  pattern  X  submatrices  occupy  the  upper 
right  part,  centered  at  (3,8).  Since  pattern  X  is  very  different 
from  the  I,  J,  and  T  patterns  in  the  memory  space,  using  the 
slower  learning  rate,  a  =  0.000025,  pushes  the  pattern  X  sub¬ 
matrices  farther  away  to  the  upper  right  comer,  centered  at  ( 1 ,8) 
after  800  iterations,  as  shown  in  Fig.  9(b).  In  this  figure,  we 
see  that  the  memory  space  becomes  more  organized  topologi¬ 
cally. 

When  the  similar  patterns  are  presented  at  the  input  of  a  neural 
network,  a  new  pattern  may  occasionally  override  the  old  pattern 
through  selecting  the  same  optimum  position  in  the  memory 
matrix  space,  with  the  old  pattern  gradually  fading  away.  This 
phenomenon  is  rather  similar  to  the  learning  process  of  humans: 
if  one  moves  to  Southern  California  for  a  long  period  of  time, 
he  may  forget  the  bitterness  of  the  cold  winters  in  the  northeast 
region  of  the  United  States. 

An  example  of  this  phenomenon  has  been  simulated  on  com¬ 
puter,  as  shown  in  Fig.  10.  An  input  pattern  B  is  first  presented 
to  the  optical  neural  network.  The  memory  matrix  adapts  the 
pattern  around  the  lower  right  comer,  centered  around  (8.8),  as 
shown  in  Fig.  10(a).  A  second  pattern  E  is  presented  to  the 
neural  network  at  a  later  time.  Since  pattern  E  is  similar  to 
pattern  B,  it  picks  the  same  spot  (8,8)  in  the  memory  matnx 
space  as  its  maximum  output  point.  As  can  be  seen  in  Fig.  10(b). 
the  memory  submatrices  of  B  are  eventually  taken  over  by  E. 
here  after  100  iterations.  If  one  would  like  to  preserve  the  old 
memory  while  ieaming  new  knowledge,  there  are  two  ways  to 
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Fig.  9.  Memory  matrix  topology  using  linear  learning  speeds,  (a)  Fast  learning 
rate  with  the  centers  of  the  similar  patterns  I  at  (5,1),  J  at  (7,2),  and  T  at  (8,4) 
and  the  center  of  the  different  pattern  X  at  (3,8).  (b)  Slow  learning  rate  with 
the  pattern  X  pushed  farther  away  to  the  upper  right  comer,  centered  at  (1,8). 
Notice  that  the  memory  space  becomes  topologically  more  organized  and  also 
appears  smoother. 


achieve  this.  One  way  is  to  refresh  the  memory  by  repeatedly 
reviewing  the  previous  input  patterns,  which  takes  a  lot  of  com¬ 
putational  time  and  may  cause  confusion.  Another  is  to  set  a 
rule  in  the  learning  process:  the  new  pattern  would  not  take  the 
output  nodes  within  certain  regions  of  the  old  patterns  in  the 
memory'  matrix,  i.e.,  their  center  spots.  This  rule  can  be  ex¬ 
pressed  as  follows: 


the  self-organizing  optical  neural  network  has  the  ability  to  de¬ 
velop  new  categories  in  the  memory  matrix  space  to  learn  un¬ 
known  objects  while  performing  pattern  recognition  operations. 
The  optical  neural  network  is  also  capable  of  organizing  a  feature 
map  and  preserving  old  memory  while  learning  new  knowledge. 
These  abilities  can  be  achieved  by  setting  new  criteria  of  match¬ 
ing  scores,  using  a  slower  learning  speed,  and  introducing  the 
concept  of  forbidden  regions  in  the  associative  memory  space. 


dAi„)  =  max  < 
i/.»>e*,un) 


N  N  j 

y.  2  ttl ll[ij{tn\Xij{tn)  | 
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where  ReU„)  consists  of  the  output  nodes  of  dc(t\),  dc(tz ) . 

dc(t„- 1)  and  their  neighboring  regions,  illustrated  in  Fig.  10(c). 
The  radii  rf  of  the  forbidden  region  can  be  determined  by  ex¬ 
perience  (in  our  experiments,  r(  =  2  was  selected).  After  this 
rule  is  added  to  the  Kohonen  model,  the  neural  network  can 
easily  remember  1 1  patterns  without  erasing  similar  patterns,  as 
shown  in  Fig.  10(d).  The  centers  of  these  11  patterns  in  the 
memory  matrix  are  B  at  (8,4),  C  at  (8,6).  E  at  (6,8),  F  at  (8,8), 

I  at  (1,8).  J  at  (1.6),  T  at  (3,8),  V  at  (7,1),  W  at  (6,3),  X  at 
(1,1),  and  Y  at  (1,3). 

Although  results  can  be  obtained  in  the  self-organizing  optical 
neural  network  by  adjusting  the  learning  parameters,  the  best 
results  were  achieved  through  experience.  In  addition,  since  the 
original  Kohonen  model  was  not  designed  for  pattern  recogni¬ 
tion.  the  ability  for  self-organization  and  adaptive  pattern  rec¬ 
ognition  can  be  enhanced  by  incorporation  with  more  effective 
models,  such  as  the  interpattem  association  (IPA)  mode!,18  in 
constructing  the  interconnection  weight  matrix  during  the  learn¬ 
ing  process. 

5.  CONCLUDING  REMARKS 

We  have  demonstrated  that  the  Kohonen  self-organizing  feature 
map  algorithm  can  be  implemented  in  an  optical  neural  network 
to  perform  adaptive  pattern  recognition.  This  model  is  modified 
and  combined  with  several  matching  criteria  we  have  developed. 
Experimental  and  computer-simulated  results  have  shown  that 
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Fig- 10.  Topological  memory  matrix  space  showing  learning  without 
erasing  the  old  memory:  (a)  Pattern  B  in  the  memory  matrix  space, 
lb)  Pattern  B  submatrices  are  erased,  after  pattern  E  is  learned. 
(c>  Forbidden  regions  in  memory  matrix  space,  (d)  Learning  new  pat¬ 
terns  without  erasing  old  ones.  The  centers  of  the  11  patterns  are 
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Experimental  demonstrations  of  real-time  color  holographic  storage  in  LiNbOj  using  a  “white  light"  laser  are  presented.  The 
geometr.  of  the  recording  setup  and  wavelength  crosstalk  are  discussed. 


Two  of  the  most  widely  used  white-iight  holo¬ 
grams  must  be  the  reflection  hologram  of  Denisvuk 
[  1  ]  and  the  rainbow  hologram  of  Benton  (2.3).  In 
reflection  hologram,  a  thickness  emulsion  of  about 
20  pm  would  have  a  wavelength  selectivity  about  £s).i 
?.=  1  /40.  which  is  high  enough  to  produce  color  hol¬ 
ogram  images  without  significant  color  blur.  How¬ 
ever.  the  physical  requirements  for  constructing  a  re¬ 
flection  hologram  is  rather  stringent,  which  prevents 
its  wide  spread  use  of  applications.  On  the  other 
hand,  construction  of  a  rainbow  hologram  requires 
a  narrow  slit,  for  which  the  parallax  information  of 
the  hologram  image  would  be  partly  lost. 

In  this  communication,  we  shall  demonstrate  that 
color  holograms  can  be  constructed  in  a  photore- 
fracuve  crystal  using  a  "white-light"  laser.  Since 
photorefractive  crystal  is  much  thicker  than  conven¬ 
tional  photographic  emulsion,  it  provides  3  higher 
wavelength  selectivity  such  that  the  color  blur  can  be 
minimized.  Furthermore,  the  construction  of  pho¬ 
torefractive  holograms  is  in  reai-time  mode  and  the 
shrinkage  of  the  emulsion  can  be  prevented.  As  in 
contrast  with  the  photographic  film,  multiplexing 
color  holograms  in  a  photorefractive  crystal  is 
possible. 

By  applying  the  coupled  wave  theory  {4  j  in  thick 
emulsion  hologram,  as  illustrated  m  fig.  I.  wave- 


la)  (b) 

Fig.  !.  Writing  sngie.  ( a >  Transmission  hologram,  (b)  Reflec¬ 
tion  hologram. 

length  selectivities  for  transmission  and  reflection 
hologram  can  be  shown  as  [  5  ] 

( \  _  (r- sin:a):/:  A  ( ( ^ 

\  A  /,  sin;a  d ' 

and 

£)m  ■-  V~:,.  (2) 

I,  trr-cos-a) 

where  a  is  the  incident  angle  and  t]  is  the  refractive 
index  of  the  hologram.  The  i-or.naiized  wavelength 
sclecttviiies  as  a  function  of  •'  dent  angle  arc  plot¬ 
ted  in  fig.  2.  where  we  notice  that  the  wavelength  sc- 
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Fig.  2.  Wavelength  selectivity  normalized  by  (X/d)i]s2.2 8.  (a)  Transmission  hologram  (the  scale  on  right).  <b)  Reflection  hologram 
(the  scale  on  left). 


iectivity  for  reflection  hologram  is  one  to  two  orders 
of  magnitude  higher  than  the  transmission  hologram. 

For  a  true  color  hologram  construction,  a  red.  a 
green,  and  a  blue  wavelength  (e.g.,  630  nm.  510  nm. 
and  440  nm )  should  be  used,  in  which  the  relative 
wavelength  spacing  is  about  1/5.  Thus  the  wave¬ 
length  selectivity  of  a  color  hologram  should  be 
smaller  than  the  relative  wavelength  spacing  of  the 
light  source.  For  instance,  using  30c  incident  angle 
for  a  transmission  hologram,  the  thickness  of  the 
emulsion  should  be  greater  than  50/.  which  is  on  the 
order  of  a  few  hundred  microns.  This  thickness  can 
easily  be  obtained  by  using  photorefractive  crystal. 

To  record  a  color  transmission  hologram,  we  used 
3  1  mm  thick  LiNbOj  photorefractive  crystal  with  a 
writing  angle  of  about  15=.  To  reduce  the  noise  dis¬ 
turbances  caused  by  the  defects  of  the  crystal,  a  piece 
of  ground  glass  be  placed  at  the  front  of  a  color  ob¬ 
ject  transparency.  The  recording  light  source  is  a 
Lexei  Kr-Ar  “white-light”  laser,  which  has  nine  spec¬ 
tral  lines  distributed  within  the  visible  region.  The 
strongest  spectral  lines  of  this  laser  are  being  *83  nm. 
514  nn.  and  647  nm  wavelengths.  Since  the  spectral 
lines  os  .' "ate  at  different  transversal  modes,  spatiai 
filters  arc  used  in  both  the  object  and  the  reference 


beams  to  insure  the  color  uniformity.  The  object-to- 
refercnce  beam  ratio  in  our  experiment  is  about  1 :8, 
for  which  a  very  high  quality  color  hologram  is  con¬ 
structed.  Fig.  3  shows  a  color  hologram  image  re¬ 
constructed  using  the  ‘white-light”  laser,  notice  that 
both  vertical  and  horizontal  parallaxes  are  pre¬ 
served.  Furthermore,  by  viewing  the  hologram  im¬ 
age  at  different  angles,  it  does  not  cause  any  change 
in  *he  color,  as  in  contrast  with  the  rainbow  holo¬ 
graphic  imaging. 

We  stress  that,  excellent  color  fidelity  can  always 
be  obtained  by  properly  controlling  the  exposure  and 
spectral  lines  polarizations.  Nevertheless,  if  the  ex¬ 
posure  is  incorrect,  color  saturation  dominated  by 
the  strongest  spectral  line  would  occur  in  the  holo¬ 
gram  image. 

Multiplexing  construction  is  a  means  of  high  den¬ 
sity  holographic  storage,  for  which  the  thickness  of 
photorefractive  crystal  makes  it  possible.  Fig.  4  shows 
an  optical  setup  for  the  multiplexing  coior  hologram 
construction  process  using  a  LiNbOj.  A  condensing 
lens  L,  is  used  to  focus  the  object  beam  onto  the 
crystal.  The  direction  of  the  object  beam  can  be  ad¬ 
justed  by  positioning  l,,  by  which  vanous  object 
transparencies  can  be  multiplexing  into  the  photo- 
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Fit-  3.  Reconstructed  color  hologram  image. 


refractive  crystal.  In  the  noiogram  image  reconstruc¬ 
tion  process,  a  pinhole  is  piaced  at  P,  for  the  selec¬ 
tion  of  the  hologram  images,  which  would  be  imaged 
onto  plane  P;  by  L:.  In  our  demonstration,  we  have 
multiplexed  four  color  holograms  in  the  LiNbO, 
crystal.  We  note  that,  multi-exposure  technique  f  3] 
can  be  used  to  achieve  the  same  diffraction  effi¬ 
ciency  tor  each  color  hologram  image.  Furthermore, 
the  size  of  the  pinhole  used  at  Pi  is  about  m>.Fia. 
where  mxmis  the  number  of  pixel  of  the  object,  a 
is  the  size  of  the  object,  and  F  is  the  focal  length  of 


F. 


for  rnuitipIfUng  coior  holcfrsphic  i 


Li-  It  is  therefore  apparent  that  if  an  .Vx  A'  pinhole 
array  is  used,  then  A'1  objects  can  be  multiplexed  in 
the  photorefractive  crystal.  Since  the  pinnoic  array- 
can  be  realized  by  using  a  programmable  spatial  light 
modulator,  it  is  clear  that  the  hologram  'mages  can 
be  randomly  addr.'  sed  during  ihe  reconstruction 
process. 


in  t.,e  coior  noiogram  recording  process,  we  have 
found  that  coior  crosstalk  occurs.  :.e..  the  grating 
formed  at  one  wavelength  is  read  out  by  another 
wavelength.  The  wavelength  seiectivmes  of  cos  t  i  i 
and  s  2  \  are  essentially  the  width  of  the  main  lobe  of 
ihe  trequency  detuning  espouse.  However,  for  the 
dynamic  two-wave  mixing,  the  weak  sidelobes  would 
have  caused  serious  color  crosstalk.  Consequent!'-, 
the  color  crosstalks  are  cascaded  which  will  deteri¬ 
orate  the  reconstructed  image  quality.  The  color 
crosstalk  can  be  interpreted  as  the  spatial  beating 
phenomena.  In  tact,  we  ha>c  observed  the  beating 
frequencies  up  to  the  fifth-order. 

t-tg.  '  snows  a  wave  vector  diagram  so  represent 
the  oeating  process.  I  he  writing  sectors  <  )A  and  OC 
a.c  produces  b;  wavelength  / -white  08  and  OD 
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Fig.  S.  Wavevector  diagram  to  represent  color  crosstalk. 

are  resulted  from  A;.  They  form  grating  vectors  AC 
and  BC  respectively.  If  the  read-out  vector  is  par¬ 
alleled  with  respect  to  vectors  OA  or  OB.  then  the 
grating  vector  BD  can  oniy  be  read  out  by  How¬ 
ever.  if  the  wavelength  selectivity  of  the  hologram  is 
not  high  enough,  or  due  to  the  response  from  the  si- 
dciobes.  then  the  grating  vector  BD  can  also  be  read¬ 
out  by  A,.  In  fact,  the  read-out  vector  due  to  cros¬ 
stalk.  can  be  interpreted  as  the  conservation  of  ( wave 
vector)  .v-omentum  along  the  direction  of  the  grat¬ 
ing  vector.  Thus  by  translating  vector  BD  to  AE.  and 
drawing  EF.  parallel  to  the  bisector  of  angle  (20), 
and  intersecting  tee  circle  at  F.  a  new  wavevector  OF 
can  be  drawn.  We  have  also  seen  that  the  wavevec- 
tors  OC  and  OF  four,  a  beating  grating  CF.  which 
can  be  read  out  by  vector  OD  of  A;.  Similarly,  a  new 
wavevector  OG  is  produced  by  A>  at  about  the  same 
direction  ot  OF.  Since  vectors  Of  and  OG  would  in¬ 
teract  with  OA  and  OB.  it  woula  &«.neraie  a  second¬ 
ary  set  of  beatings.  The  angle  of  the  beating  grating 
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can  be  computed  as  follows, 

J=2  tan  &(/.,//.,  — I ) .  (3) 

We  stress  that,  fcr  small  9.  the  angles  of  beatings 
would  be  about  the  same  in  different  orders. 

Fig.  6  shows  the  cascading  beating  phenomenon 
generated  by  multi-color  two-beam  mixing  in  a 
LiNbOj  photorefractive  crystal  of  1  mm  thick,  where 
the  interference  angle  26  is  set  about  1 5 : .  The  bright¬ 
est  spot  shown  in  this  figure  represents  the  normal 
reconstructed  spot,  while  the  other  spots  are  gener¬ 
ated  from  the  beating  phenomenon  due  to  color 
crosstalks.  The  color  crosstalks  are  primarily  caused 
by  the  blue  light  gratings  read-out  by  green  wave¬ 
length  or  vice  versa.  For  example,  the  angle  of  the 
beating  grating,  J.  of  the  right-side  spots  is  measured 
to  be  0.0 103  ±0.001  %  which  is  in  agreement  with 
the  calculated  result  of  0.0 1 02 : .  as  obtained  from  eq. 
(3).  Similarly,  the  spots  on  the  left-side  are  gener¬ 
ated  by  the  green  beating  gratings  read-out  by  blue 
light. 

Wfe  have  demonstrated  color  holographic  imaging 
using  a  thick  crystal  LiNbOj  photorefractive  crystal 
with  a  “white-light’*  laser.  We  have  experimentally 
shown  that  high  quality  color  hologram  image  can  be 
obtained  with  a  thick  crystal  without  sacrificing  the 
parallax.  Multiplexing  color  holograms  in  a  thick 
photorefractive  crystal  is  also  discussed,  for  which 
we  have  proposed  using  a  programmable  spatial  light 
modulator  for  the  random  address.  Color  crosstalk 
caused  by  the  grating  beating  phenomena  is  also 
studied.  To  minimize  the  color  crosstaik.  a  higher 
wavelength  selectivity  (i.e..  thick  crystal)  photore- 
Tractive  media  than  normally  required  should  be 
used. 


rig.  6.  Grating  beating  pr.enotr.enon. 
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An  optical  neural  neticork  using  two  tightly  cascaded 
liquid  crystal  televisions  is  presented.  This  new  optiesi 
architecture  offers  compactness  in  size,  ease  of  alignment, 
higher  light  efficiency',  better  image  quality ,  and  lac  cost. 
The  implementation  of  the  autoassociative  and  ketemasso- 
dative  memories  is  given. 

Much  work  has  been  done  i-»  bring  about  the  optical  imple¬ 
mentation  of  neural  network.1-7  primarily  ca  the  parallelism 
and  massive  intercoanectivity  of  optics.  The  first  2-D  opti¬ 
cal  neural  network  (2-D  ONN)  to  simulate  the  Hopfieid 
model  was  proposed  by  Far  hat  and  Psaitisin  1936.:  An  LED 
array  was  used  as  the  input  feature  composer,  which  is  opti¬ 
cally  interconnected  to  a  weighting  mask,  oiled  the  inter¬ 
connection  weight  matrix  (IWM)  or  associative  memory  ma¬ 
trix  by  a  lensiet  array.  To  provide  the  neural  network  with 
learning  capability,  a  fine  resolution  and  high  dynamic  range 
programmable  spatial  light  modulator  (SLMi  are  required 
for  the  generation  of  the  fWM, 

Recently  we  proposed  a  different  ONN  architecture  using 
a  high  resolution  video  monitor*4  by  which  the  interconnec- 
tive  part  in  the  Hopfieid  equation  can  be  optically  added. 
More  recently,  *e  constructed  an  ONN  using  liquid  crystal 


LCTVI  LCTV2 


Fig,  I.  Compact  ONN  architecture  using  cascaded  LCTV*. 


televisions  (LCTVs).7  In  view  or  the  preceding  ONN  archi¬ 
tectures.  a  large  numerical  aperture  tmapsg  lens,  of  the 
order  of  1/0.7.  I*  needed  to  image  the  ienslet  array  onto  the 
output  charge-coupled  device  (CCD)  detector.  The  ardri- 
ttc'ure-j  suffer  from  low  light  efficiency,  high  aberration,  and 
larger  she  due  to  the  imaging  less. 

To  aUeriate  these  problems,  we  propose  a  new  compact 
architecture  using  tightly  cascaded  uCTVs.  as  shown  in  Fig. 
I.  Let  us  recall  the  iterative  operation  c.  a  2-D  NxNtmmm 
network.  It  can  be  described  by  a  matrix-vector  product 
operation  of  Hopfieid  as  given  by* 
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Fig.  2.  Display  format  of  a  2-D  input  pattern  and  4-D  IWM:  (a) 
input  pattern;  (b)  IWM. 
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Fig.  3  Experimental  result  for  an  autoassociative  memory,  (a) 
four  capital  letters  stored  in  the  IWM,  (b)  positive  IWM;  (c)  negative 
IWM,  (d)  partial  input  pattern;  (»>  reconstructed  pattern. 


where  Vtk  and  Ut,  represent  the  state  of  the  Ik th  and  ijth 
neuron  in  the  N  X  N  neuron  space,  T,kl/  is  a  4-D  IWM,  and  n 
represents  the  nth  iteration.  Notice  that  the  matrix  T  can 
be  partitioned  into  an  array  of  2-D  submatrices  T(*„,  T/*i2, 
....  and  TikNNi  in  which  each  submatrix  is  of  the  NX  N  size. 
Thus  the  PVM  can  be  displayed  as  an  N2  X  N2  array 
representation. 

By  referring  to  the  proposed  architecture  of  Fig.  1,  the 
input  pattern  and  the  IWM  are  displayed  on  the  cascaded 
LCTVs,  as  illustrated  in  Fig.  2.  The  light  emerging  through 
the  cascaded  LCTVs  is  apparently  proportional  to  the  prod¬ 
uct  of  TikuU^n)  inEq.  (1).  It  on  be  seen  that  each  subma- 
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Fig.  4.  Experimental  result  for  heteroassociative  memory:  (a)  in- 
put-output  training  set;  (b)  positive  IWM;  (c)  negative  ’  VM;  (d) 
partial  input  pattern;  (e)  output  pattern. 

trix  T uujU„(n)  is  superimposingly  imaged  onto  the  CCD 
detector  by  the  lenslet  array.  Thus  the  output  intensity 
array  from  the  CCD  detector  is  the  summation  of  the  subma¬ 
trices  T ikijUijin)  over  i  and  j.  We  stress  that  the  summation 
is  essenti.-T'y  carried  out  optically  by  the  lenslet  array. 
Needless  to  say  that,  by  thresholding  the  signals,  the  result 
oan  be  fed  back  to  the  input  LCTV  via  a  microcomputer  for 
-he  next  iteration.  Therefore,  a  closed  loop  operation  can  be 
obtained  with  the  proposed  system.  Furthermore,  with  the 
elimination  of  the  imaging  lens,  the  system  can  be  built  in 
compact  size. 

Moreover,  by  cascading  the  input  pattern  with  the  IWM. 
the  output  pattern  pixels  from  the  ONN  remain  of  the  same 
shape  as  the  input  pattern  instead  of  circular  shape  as  in  the 
previous  architectures. 

An  8X8  neuron  neural  network  using  a  cascaded  Hitachi 
12. /-cm  (5-in.)  color  LCTVs  is  built  in  the  Electro-Ootics 
Laboratory  at  the  Pennsylvania  State  University.  The  reso¬ 
lution  of  the  LCTV  is  240  X  480  pixels  with  a  contrast  ratio  of 
above  15:1  under  white  light  ilium;  ion. 

In  the  experimental  demonstrations,  we  have  used  the 
interpattern  association  (IPA)  mclel  for  the  construction  of 
the  IWM.8  Both  autoassociative  and  heteroassociative 
IWMs  have  been  used  in  the  ONN.  An  autoassociative 
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IWM  for  the  English  letters  A,  B,  C,  and  D  is  shown  in  Figs. 
3(b)  and  (c).  If  a  partial  pattern  of  A,  as  illustrated  in  Fig. 
3(d),  is  presented  at  the  input  of  the  ONN,  a  recovery  of 
patte-n .  t  is  obtained  after  one  iteration  in  our  experiment, 
a«  s:..>wr.  in  Fig.  3(e). 

F«,r  the  Heteroassociative  memory,  four  English  letters  and 
four  Chinese  characters  are  used  as  the  input-output  train¬ 
ing  set,  as  shown  in  Fig.  4(a).  The  heteroassociative  IWM  is 
depicted  in  Figs.  4(b)  and  (c).  Again,  if  a  partial  a  is  present¬ 
ed  at  the  input  of  the  ONN,  the  corresponding  Chinese 
character  is  obtained  after  one  iteration,  as  shown  in  Fig. 
4(e). 

In  conclusion,  we  built  a  compact  ONN  using  tightly  cas¬ 
caded  LCTVs.  This  optical  architecture  offers  the  advan¬ 
tages  of  compact  size,  easy  alignment,  higher  light  efficiency, 
better  image  quality,  and  low  cost.  Demonstrate '  is  of  the 
autoassociative  memory  and  heteroassociative  nslation 
are  provided. 
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The  wavelength  multiplexing  properties  of  a  reflection  filter  are  investigated.  A  photorefraenve  LiNbO,  crystal  was  used  to 
construct  an  experimental  filter.  The  wavelength  selectivity  and  shift  invariant  properties  of  the  filter  were  analyzed  and  experi¬ 
mentally  verified. 


Recently,  multiplexing  of  filters  in  photorefractive 
materials  has  attracted  much  attention.  By  measur¬ 
ing  the  maximum  phase  shift  achieved  in  a  deep  vol¬ 
ume  hologram,  and  calculating  an  exposure  schedule 
for  recording  multiple  equally  diffracting  holograms, 
Hong  et  al.  have  estimated  that  over  1000  holograms 
could  be  recorded  in  a  4  mm  thick  BaTiOj  crystal 
[1  ].  The  holograms  can  be  recorded  in  the  crystal 
using  angular  or  wavelength  multiplexing  tech¬ 
niques.  By  using  a  fixed  image  position,  and  chang¬ 
ing  the  angular  position  of  the  reference  beam,  Mok 
et  al.  have  reported  recording  500  high  resolution, 
uniformly  diffracting  volume  holograms  in  a  single 
LiNb03  crystal  [2].  By  writing  holograms  sequen¬ 
tially,  and  changing  the  writing  wavelength  between 
exposures,  multiple  holograms  can  be  recorded  in  the 
crystal.  This  wavelength  multiplexing  technique  has 
the  advantage  that  neither  the  object  nor  the  refer¬ 
ence  beam  needs  to  be  changed  in  the  writing  or 
reading  processes,  even  though  the  two  processes  can 
be  used  in  conjunction  to  further  increase  the  mul¬ 
tiplexing  capabilities.  This  technique  has  been  used 
in  holographic  film  in  which  matched  fillers  were 
multiplexed  (3].  However,  the  thin  film  emulsion 
leads  to  low  wavelength  selectivity,  and  the  filters  are 
difficult  to  produce.  In  addition,  the  system  was  not 
capable  of  quick  updating  of  the  reference  patterns. 

In  this  paper  we  will  investigate  wavelength  mul¬ 
tiplexed  matched  spatial  filters  using  a  LiNbOj  crys¬ 


tal.  Referring  to  the  coupled-mode  analysis  of  Ko- 
gelnik  [4],  we  will  analyze  the  wavelength 
multiplexing  capacity  of  reflection  type  matched 
spatial  filters.  By  integrating  over  the  thickness  of  the 
crystal,  as  was  suggested  by  Gheen  and  Cheng  [5], 
we  will  show  that  thick  crystal  filters  exhibit  high 
wavelength  selectivity  as  well  as  retaining  a  good  de¬ 
gree  of  the  shift  invariance.  Thus,  by  recording  single 
wavelength  filters,  and  reading-out  the  filters  with  a 
multi-wavelength  source,  parallel  pattern  recogni¬ 
tion  can  be  performed. 

It  is  well  known  that  volume  holograms  offer  high 
diffraction  efficiency  and  high  angular  and  wave¬ 
length  selecttvities.  Reflection  volume  holograms 
have  a  much  higher  wavelength  selectivity  than 
transmission  volume  holograms,  while  the  angular 
selectivity  of  the  holograms  remain  similar.  For  this 
reason,  we  chose  to  use  reflection  volume  holograms 
for  the  matched  filter  synthesis.  Using  the  weak  cou¬ 
pling  approximation,  where  multiple  diffractions  are 
ignored,  the  wavelength  selectivity  of  the  reflection 
hologram  can  be  expressed  as  (6) 

f^)=  ,  v-—  r  4  ■  (1) 

\  *-  /,  v  n'-cos*a£> 

where  A  is  the  central  waselcngth  of  the  writing  beam, 
D  is  the  thickness  of  the  crystal,  n  is  the  refractive 
index  of  the  cry  stal,  and  a  is  the  external  angle  shown 
in  fig.  1. 
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Fig.  1.  Writing  angle  for  reflection  hologram. 

The  shift  invariant  property  of  the  spatial  filter  can 
be  determined  using  layer  integration  over  the  thick¬ 
ness  of  the  crystal  as  proposed  by  Gheen  and  Cheng 
{5].  Fig.  2  shows  a  schematic  diagram  for  the  syn¬ 
thesis  of  a  reflection  type  matched  spatial  filter. 
Without  loss  of  generality,  we  will  use  a  1-D  repre¬ 
sentation  for  the  analysis,  in  which  <7,  (*, )  represents 
the  reference  image  that  is  located  at  the  front  focal 
plane  of  the  transform  lens  L.  For  a  given  point  ,v,, 
a  wave  represented  by  wavevector  *,(*,.  A,)  is  in¬ 
cident  at  the  photorefractive  crystal.  The  reference 
beam  is  a  plane  wave  represented  by  wavevector 
Ac0(A,),  and  is  incident  on  the  opposite  side  of  the 
crystal.  Thus,  it  is  apparent  that  a  reflection  type 
matched  filter  can  be  recorded  in  the  crystal.  Let 
q2{x2)  be  the  readout  beam  representing  the  input 
object,  which  is  to  be  detected.  Thus,  each  point  x2 
would  produce  a  readout  wavevector  k2(x2,  A2)  for 
which  the  reconstructed  wavevector  is  *3(^3,  A2), 
where  x}  denotes  the  output  coordinate  plane.  The 


wavevector  representations  in  the  crystal  can  be  ex¬ 
pressed  as. 

*o(Ai)  =  — (rt/A|)z, 


where  z  and  A  are  the  unit  vectors  along  and  normal 
to  the  optical  axis  of  the  system,  respectively.  The 
output  correlation  signal  can  be  expressed  by  the  fol¬ 
lowing  integral  equation: 

K(Xj)=/ljJJJJ  qmx(xx)  q2(x2) 

2 

Xexp[i27tM  (uu+zz)]  d*,  dx2d«dz  ,  (3) 

where  Ak  represents  the  Bragg  dephasing  vector, 

Ak=k0-k[  +k2-k2 .  (4) 

For  simplicity,  assume  that  the  size  of  the  crystal  is 
infinite  in  the  A  direction,  therefore  the  A  component 
of  the  Bragg  dephasing  wavevector  must  be  zero. 
From  eq.  (4),  we  have, 

.v,/A, -(.v2+.v3)/A2=0  .  (5) 


Fig  2.  Geometry  for  reflection  matched  filter  x,.  x:.  and  x,  arc  reference,  input,  and  output  plane,  respectively  kn.  k,.  k2,  and  k,  arc 
reference,  writing,  reading,  and  readout  wavevcctors  respectively 
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It  follows  that 


x2  =  ( 1  —  AA/A i )x,  — x3 ,  (6) 

where  AA= A ,  —  A2.  Since  the  relative  wavelength  dif¬ 
ference,  AA/A ,  is  small,  the  third  order  terms  can  be 
neglected.  Thus,  the  i  component  of  the  Bragg  de¬ 
phasing  vector  can  be  written  as 

M.=  — [2n2^  — .v3U, -.v3)  1.  (7) 

«*i  La,  J 

By  substituting  eqs.  (6)  and  (7)  into  eq.  (3),  and 
integrating  with  respect  to  u.  and  x2,  the  correla¬ 
tion  signal  is 

R(Xj)=A  jj  ??(x,)  ^2[(1_  y)*' 

Xsinc|j^-^2n2 y-  -x3(x,  -x3)^Jdx,|  .  (8) 

With  reference  to  the  sine  factor,  one  can  evaluate 
the  wavelength  selectivity  and  the  shift  invariant 
properties  of  the  thick  crystal  filter.  If  the  input  ob¬ 
ject  is  centered  with  respect  to  the  reference  function 
(i.e..  x,=x2),  then  the  filter  would  respond  to  the 
wavelength  variations  that  satisfy  the  inequality  of 
eq.  (9), 

|  AA/A|  |  <A,/nD.  (9) 


This  result  is  essentially  the  same  as  was  obtained 
with  coupled  wave  theory  of  eq.  ( 1 )  when  the  beams 
are  perpendicular  to  the  surface  of  the  crystal. 

On  the  other  hand,  if  the  read-out  and  the  writing 
wavelength  are  identical,  then  the  filter  would  re¬ 
spond  to  object  shifting  that  satisfies  the  inequality 
of  eq.  (10), 


bc3(x,  -x3) 


F1 


Al 

nD' 


(10) 


where  F  represents  the  focal  length  of  the  transform 
lens. 

By  substituting  eq.  (6)  into  eq.  (10),  and  letting 
x2=x,  -  Ax,  we  have 

|  Ax(X|  —  Ax)  |m„  </.\F1/nD .  (11) 

This  inequality  shows  that  the  allowable  shift.  Ax.  is 
dependent  on  the  halfwidth  of  the  input  object 
(x,  -  Ax)m„,  the  thickness  of  the  crystal,  and  the  fo¬ 


cal  length  of  the  lens.  It  follows  that  the  wavelength 
selectivity  and  shift  invariance  property  are  depen¬ 
dent  on  the  thickness  of  the  crystal.  In  addition,  by 
changing  the  effective  focal  length  F  of  the  system, 
a  desired  shift  invariance  for  a  given  thickness  of  the 
crystal  can  be  obtained  [7], 

Fig.  3  shows  the  optical  setup  that  was  used  for  the 
experiments.  The  beam  splitter  separates  the  beam 
into  a  reference  beam  that  enters  the  crystal  from  the 
right,  and  an  object  beam  that  illuminates  the  input 
plane,  which  is  transformed  by  lens  L|  ontc  a  1  mm 
thick  LiNbOj  crystal.  These  two  beams  interfere  in 
the  crystal  to  form  a  reflection  type  matched  filter. 
After  the  filter  has  been  recorded,  the  reference  beam 
is  blocked,  and  an  input  object  to  be  correlated  with 
the  filter  is  placed  in  the  input  plane.  The  light  scat¬ 
tered  by  the  filter  forms  an  output  signal  that  is 
transformed  by  lens  L2  onto  the  output  CCD  detec¬ 
tor,  which  is  connected  to  a  video  monitor  for  real¬ 
time  observation. 

In  the  first  experimental  demonstration,  we  tested 
the  wavelength  multiplexing  capabilities  of  a  Li- 
Nb03  crystal.  By  using  A=488  nm,  D- 1  mm,  and 
2.25  in  eq.  (9),  we  found  that  the  filter  should 
not  respond  for  wavelength  shifts  of  AA>0.106  nm. 
We  recorded  individual  matched  filters  using  wave¬ 
lengths  457.9,  465.8,  488,  and  496.5  nm.  Reading 
out  these  filters  with  wavelengths  different  from  the 
recorded  wavelength  produced  no  visible  output  sig¬ 
nal.  Next,  a  multiplexed  filter  using  the  previously 
cited  wavelengths  was  recorded  of  the  capital  letters 
D,  F,  K,  and  J.  Fig.  4a  shows  the  output  autocor- 
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Fig  4  Oulpul  of  correlator,  (a)  Autocorrelation  of  letter  D.  (b) 
Intensity  plot  of  output  from  part  a  (c)  Crosscorrelation  with 
letter  C.  using  same  wavelength. 

relation  spot  of  a  letter  D  using  wavelength  457.9  nm. 
The  corresponding  intensity  profile  of  the  autocor¬ 
relation  signal  detected  by  the  CCD  camera  is  shown 
in  fig.  4b.  In  this  figure,  we  see  that  a  high  intensity 


b 


♦ 


c 


Fig.  5  Shift  invariance  property  of  reflection  volume  filter,  (a) 
Autocorrelation  of  letter  O.  (b)  Input  object  consisting  of  two 
O’s  and  an  X.  ( c )  Output  of  correlator  using  input  shown  m  pan 
b. 

correlation  peak  was  obtained.  Fig.  4c  shows  the  out¬ 
put  cross  correlation  of  this  filter  with  an  input  ob¬ 
ject  C  using  a  wavelength  457.9  nm.  It  is  apparent 
that  no  detectable  peak  was  observed.  Thus,  we  have 
seen  that  the  wavelength  multiplexing  capabilities  of 
the  crystal  filter  are  in  good  agreement  with  our 
analysis. 
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In  the  second  experiment  the  shift  invariant  prop¬ 
erty  of  the  crystal  filter  was  verified.  By  using/.=438 
nm.  « = 2.25.  D=  1  mm.  and  effective  focal  length 
F=  1  m.  in  eq.  ( 10)  is  clear  that  [iXv  (Ay-  Ax)  ]  must 
be  less  than  216  mm:.  The  objects  that  were  used  for 
this  experiment  have  haifwidth  of  1 .3  mm.  for  which 
the  shift  invariance  expression  reduces  to  Axm,  <  1 6 
cm.  which  is  much  larger  than  the  beam  diameter.  A 
reflection  type  filter  of  capital  letter  O.  centered  at 
the  input  plane  was  recorded.  The  corresponding 
output  autocorrelation  signal  is  shown  in  fig.  5a. 
Next,  a  transparency  consisting  of  two  O’s  and  an  X 
as  shown  in  fig.  5b  was  inserted  at  the  input  plane  of 
the  optical  system.  The  letters  were  translated  about 
9  mm  with  respect  to  the  position  of  the  reference 
object  of  the  filter.  The  corresponding  output  cor¬ 
relation  spots  are  shown  in  fig.  5c.  where  two  strong 
correlation  spots  are  observable,  as  well  as  a  weak 
cross-correlation  distribution.  The  input  beam  in  this 
experiment  had  a  usable  radius  of  about  ~  1.2  cm. 
and  beyond  this  radius  the  beam  exhibited  rapid  am¬ 
plitude  variations.  Next,  the  input  object  was  shifted 


within  the  beam,  and  we  found  that  very'  little  am¬ 
plitude  variation  m  the  correlation  peak  intensity  was 
observed.  As  before,  the  experimental  results  are  in 
good  agreement  with  our  analysis. 

We  acknowledge  the  support  of  the  NASA  Re¬ 
search  Grant  NAG9-403/Basic. 
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A  space-time-sharing  optical  neural  network  for  implementing  a  large-scale  operation  is  presented.  If  tne  intercon¬ 
nection  weight  matrix  is  partitioned  into  an  array  of  submatrices,  a  large  space-bandwidth  pattern  can  be  processed 
with  a  smaller  neural  network.  We  show  that  the  processing  time  increases  as  a  square  function  of  the  space- 
bandwidth  product  of  the  pattern.  To  illustrate  the  space-time-sharing  operation,  experimental  and  simulated 
results  are  provided. 


The  massive  interconnection  and  parallel  processing 
capabilities  of  optics  offer  advantages  for  optical  com¬ 
puting.  In  recent  years,  much  effort  has  been  applied 
for  the  optical  implementation  of  neural  networks.1-10 
To  provide  the  neural  network  with  learning  ability, 
spatial  light  modulators  (SLM’s),  such  as  a  video  mon¬ 
itor*-8  and  a  liquid-crystal  television9-10  (LCTV),  have 
been  used  as  the  programmable  devices.  In  a  fully 
interconnected  neural  network,  every  neuron  at  the 
input  plane  is  connected  to  all  the  neurons  at  the 
output  plane.  For  example,  1000  neurons  would  re¬ 
quire  a  million  interconnections.  Thus  an  extremely 
high-resolution  SLM  is  required  for  the  massive  inter¬ 
connection.  However,  the  resolution  of  the  currently 
available  SLM’s  is  rather  limited.  This  poses  an  ob¬ 
stacle  in  developing  an  optical  neural  network  for 
large-scale  operation.  In  this  Letter  we  use  a  space- 
time  sharing  technique  to  alleviate  this  constraint. 

For  an  N  X  N  neuron  network,  the  interative  equa¬ 
tion  can  be  described  by2 


IWM  cannot  be  fully  displayed  onto  LCTVl.  We 
discuss  in  the  following  cases  how  a  small  neural  net¬ 
work  may  accommodate  a  large-size  IWM: 

For  iV2  >  R  and  LN  <  R,  we  let  D  -  inUN/L),  where 
int(-)  is  the  integer  function.  The  IWM  can  be  parti¬ 
tioned  into  D  x  D  sub-IWM’s,  and  each  sub-IWM 
consists  of  LX  L  submatrices  of  NX  N  size,  as  shown 
in  Fig.  2.  To  complete  the  iterative  operation  of  Eq. 
(1),  D  X  D  sequential  operations  of  the  sub-IWM’s  are 
required.  Thus  a  smaller  neural  network  can  handle  a 
larger  space-bandwidth  product  (SBP)  input  pattern 
through  sequential  operation  of  the  sub-IWM’s. 

For  N2>R  and  LN  >  R,  we  let  D  —  int {NIL)  and  d  = 
mULN/R).  In  this  case  the  submatrices  within  the 
sub-IWM’s  are  further  divided  into  d  X  d  smaller 
submatrices,  and  each  submatrix  is  (N/d)  x  (N/d)  in 
size,  as  shown  in  Fig.  3.  The  iterative  equation  can  be 
written  as 

VaO*  +  1) 


Vlk(n  +  1)  =  f 

(I) 

-t III 

f  T  TlitlUJn) 

(jj=0s«0 

:»pd+l  j—qdTl  _  J 

where  n  denotes  the  nth  iteration,  f(-)  represents  a 
nonlinear  operator,  l7/*  and  U,,  represent  the  state  of 
the  Ik th  and  i/th  neurons,  respectively,  and  Tikis  is  the 
connection  strength  from  the  Ik  th  to  the  i/th  neuron. 

is  known  as  the  interconnection  weight  matrix 
(IWM)  and  can  be  partitioned  into  an  array  of  N  X  N 
submatrices.2-8 

An  optical  neural  network  (Fig.  1)  is  used  in  our 
discussion,  in  which  the  IWM  and  the  input  pattern 
are  displayed  onto  LCTVl  and  LCTV2,  respectively. 
Each  lens  in  the  lensiet  array  images  a  specific  subma¬ 
trix  onto  the  input  LCTV2  to  establish  the  proper 
interconnections.  The  lensiet  array  is  imaged  by  the 
imaging  lens  onto  the  output  plane.  The  overall  in¬ 
tensity  transmitted  by  each  lens  in  the  lensiet  array  is 
picked  up  by  the  charge-coupled-device  (CCD)  detec¬ 
tor  as  the  output  value.  We  let  the  resolution  LCTV  1 
be  limited  by  R  X  R  pixels,  and  the  lensiet  array  is 
equal  to  L  x  L  neurons.  If  the  size  of  the  IWM  is 
larger  than  the  resolution  of  LCTVl.  i.e..  N-  >  R ,  the 


The  input  pattern  is  also  partitioned  into  dxd  sub¬ 
matrices,  and  each  submatrix  is  (N/d)  X  (N/d)  in  size. 
Thus  by  sequentially  displaying  each  of  the  IWM  sub- 
matrices  with  respect  to  the  input  submatrices  onto 
LCTVl  and  LCTV2,  respectively,  one  can  process  an 
extremely  large  SBP  pattern  with  a  small  neural  net- 
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Fig.  1.  Schematic  diagram  of  the  optical  neural  network. 
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Fig.  3.  Partition  of  a  sub-IW  M  into  d  x  d  submatrices  (d  = 
2).  fa)  The  pqth  sub-lWM,  (b)  d  X  d  smaller  submatrices. 


work.  It  is  trivial  that  the  price  we  paid  is  prolonging 
the  processing  speed  by  D 2  x  d2  times. 

Generally,  the  processing  time  increases  as  the 
square  function  of  the  SBP,  of  the  input  pattern,  as 
given  by 

T2  =  [(AC  X  N2)/(Nl  x  ATjU2^  =  (NJh\)ATlt 

(N2  >  N,  >  =  L),  (3) 

where  7?  and  T-,  are  the  processing  times  for  the  input 
patterns  with  Nz  X  AC  and  N\  X  N<  resolution  ele¬ 
ments,  respectively  For  instance,  if  the  resolution 
elements  of  the  input  pattern  increase  lour  times  in 
each  dimension,  i.e.,  JV2  -  4A?S,  the  processing  time 
would  be  4’  =  256  times  longer. 

For  experimental  demonstrations  we  show  that  pat¬ 
terns  with  12  X  12  resolution  elements  can  be  pro¬ 
cessed  using  the  6  x  6  neuron  network  shown  in  Fig.  1. 
A  240  X  480  element  Hitachi  color  LCTV  (LCTVl)  is 
used  for  displaying  the  IWM.  Since  each  color  pixel  is 
composed  of  red,  green,  and  blue  elements,  the  resolu¬ 
tion  is  actually  reduced  to  240  X  160  pixels.  In  the 
experiment,  however,  we  used  2X2  pixels  for  each 


Fig.  5  Computer-simulated  results  of  processing  24  X  24 
eieme  it  patterns  on  a  6  x  6  optical  neural  network,  (a)  Four 
reference  patterns  stored  in  the  IWM.  (b)  the  partial  input 
pattern,  (c)  four  of  sixteen  6X6  suboutput  arrays,  (d)  the 
composed  output  pattern. 
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sub-IWM  array  and  displayed  one  by  one  onto 
LCTVl.  If  an  input  pattern  is  displayed  on  LCTV2. 
the  signals  collected  by  the  CCD  camera  can  be  thresh- 
olded  and  then  composed  to  provide  an  output  pat¬ 
tern,  which  has  a  SBP  four  times  larger  than  that  of 
the  optical  neural  network. 

One  of  the  experimental  results  is  shown  in  Fig.  4. 
The  training  set  is  shown  in  Fig.  4(a),  and  each  of  these 
patterns  is  limited  by  a  12  X  12  pixel  matrix.  Figure 
4(b)  shows  a  part  of  the  second  image  with  12  X  5 
pixels  blocked  as  the  input  pattern.  The  sub-IWM’s 
are  then  sequentially  displayed,  one  by  one,  onto 
LCTVl.  Different  parts  of  the  output  pattern  are 
obtained,  and  one  of  the  output  parts  is  shown  in  Fig. 
4(c).  The  final  output  pattern  recalled  by  the  optical 
neural  network  is  given  in  Fig.  4(d). 

To  demonstrate  the  larger-scaie  operation  further,  a 
24  X  24  neuron  IWM  is  used.  Since  N  =  24,L=6,R  = 
80,  N2  =  576  >  R,  and  LN  =  9 6>R,  we  take  D  int(AVL) 
=  4  and  d  =  int (LN/R)  —  2.  Thus  the  IWM  is  parti¬ 
tioned  into  4X4  sub-IWM’s,  and  each  sub-IWM  is 
divided  into  2x2  smaller  submatrices,  as  illustrated 
in  Fig.  3.  In  this  case  the  input  pattern  is  also  divided 
into  2X2  submatrices,  and  each  submatrix  is  12  X  12 
in  size.  It  is  apparent  that  by  sequentially  displaying 
the  submatrices  of  IWM  and  the  input  submatrices 
onto  LCTVl  and  LCTV2,  respectively,  a  24  X  24  out¬ 
put  pattern  can  be  obtained. 

Figure  5(a)  shows  four  24  X  24  pixel  images  as  the 
training  set.  A  partial  pattern  of  the  third  image, 
shown  in  Fig.  5(b),  is  used  as  the  input  pattern,  which 
is  divided  into  2X2  matrices  of  a  12  X  12  size  during 
the  processing.  Four  of  sixteen  6X6  output  parts  are 
shown  in  Fig.  5(c).  In  going  through  all  the  partitions 
of  the  sub-IWM’s,  we  compose  an  output  pattern  of 
the  image,  as  shown  in  Fig.  5(d).  The  whole  process 
takes  D~  X  d2  =  64  operations  of  the  optical  neural 
network. 


In  conclusion,  we  have  used  a  space-time-sharing 
technique  for  large-scale  nei  ron  operation.  We  have 
shown  that  to  achieve  a  large  SBP  of  the  system,  addi¬ 
tional  expenditure  of  processing  time  is  needed.  The 
amount  of  processing  time  increases  as  the  square 
function  of  the  SBP  of  the  input  pattern.  For  experi¬ 
mental  demonstration,  we  have  implemented  the  pro¬ 
cessing  of  12  X  12  element  patterns  on  a  6  X  6  neuron 
network.  The  processing  of  24  X  24  element  patterns 
on  a  6  X  6  neuron  network  has  been  further  demon¬ 
strated  by  computer  simulation. 

We  acknowledge  the  support  of  the  U.S.  Army  Mis¬ 
sile  Command  through  the  U.S.  Army  Research  Office 
under  contract  DAAL03-87-0147. 
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SLM,  correlation  operation  of  the  reference  and  the  input  targets  can  be 
obtained  A  computer  simulation  for  the  target  embedded  in  an  additive 
Gaussian  noise  is  provided. 


A  programmable  joint  transform  correlator  for  optical  pattern 
recognition  has  been  discussed  in  recent  papers  (1,  2].  In  a 
joint  transform  correlator,  both  the  reference  image  and  the 
input  target  are  generated  in  the  input  plane  of  a  coherent 
optical  processor.  A  transform  lens  will  perform  the  joint 
transform  of  the  input  objects.  Due  to  coherent  illumination, 
uniform  earner  interferometric  fringes  are  produced  in  the 
joint  transform  spectrum.  The  interfere  metric  fringes  can  be 
extracted  by  a  square  law  converter  (w.g.,  film.  CCD  camera, 
etc.).  Then  by  simple  coherent  readout  of  the  joint  power 
spectral  distribution,  the  joint  correlation  of  the  input  object 
function  can  be  obtained  through  a  second  Fourier  transform 
lens. 

In  principle,  a  joint  transform  correlator  is  capable  of 
performing  any  shift-in varia  .t  linear  operations  bv  using  the 
impulse  response  of  the  filter  function  which  is  displayed  as  a 
reference  intake  [3].  However,  for  the  convolution  operation, 
the  reference  image  must  be  the  complex  conjugate  of  the 
inverted  impulse  response  function.  With  the  current  stage  of 
technology,  the  spatial  light  modulator  (SLM)  is  applicable 
only  t  real  impulse  responses  due  to  the  lack  of  a  complex 
image  display  device.  Furthermore,  low  dynamic  range  and 
slow  response  of  the  square  law  converters  give  rise  to  another 
practical  restriction.  One  possible  realization  of  the  joint 
transform  correlator  is  to  employ  an  electrooptic  device  as  a 
square  law  converter  (e.g.,  CCD  camera,.  The  detected  joint 
transform  power  spectrum  is  then  displayed  on  a  binary  SLM 
for  the  second  transformation. 

In  this  paper,  we  propose  a  joint  transform  hybrid  optical 
correlator  using  a  binary  phase  only  filter  (4,  5).  The  system 
takes  advantage  of  the  controllability  of  the  electrooptic  de¬ 
vices  and  the  high  speed  optical  processing  operation.  As  is 
shown  in  Figure  1,  the  pioposed  joint  transform  correlator 
mvolves  gray  level  icaso.e  displaying  devices  and  a  binary 
phase  spatial  F'ght  modulator  both  of  which  can  be  addressed 
by  a  microcomputer.  The  image  displaying  devices  can  bo 
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implemented  by  using  liquid  crystal  televisions  (LCTV)  [6], 
The  phase  only  filter  can  be  synthesized  in  a  programmable 
spaual  light  modulator,  e.g.,  a  magnetooptic  spatial  light 
modulator  (MOSLM)  [5].  We  note  that  the  joint  transform 
interference  fringes  can  be  binarized  by  a  microcomputer.  The 
basic  objective  of  binarizing  the  fringes  is  to  synthesize  a 
binary  phase  only  joint  power  spectrum  that  can  be  recorded 
at  the  MOSLM. 

In  Figure  1,  we  assume  the  reference  image  /,(*,  y)  and 
the  input  target  f2(x,  y)  are  written  on  the  LCTV.  Under 
coherent  illumination,  the  complex  light  distribution  the 
Fourier  plane  P,  is  given  by 

G(p,q)  =  F[(p,q)e>a‘>  F  F2{  p,  q)c~^  (1) 

F\(p>q)  ”3F\fAx,y)}  (2) 

Fi(.p,q)  ~  y)]  (3) 

where  &  denotes  the  Fourier  transform  and  2a  is  the  center- 
to-center  separation  of  the  input  objects  /,(x,  y)  and  f2(x,  y). 
We  stress  that  the  purpose  of  the  lens  L v  is  to  magnify  the 
joint  transform  spectrum  and  that  the  stop  behind  the  lens 
selects  only  a  certain  bandwidth. 

The  image  captured  by  the  CCD  camera  can  te  written  as 

KP'<{)  “ \G(p,q)\ 2 

=*I^(P.9)12  +i^(P-9)|Z 

+  FliP,<!)F2,(P>‘l)ejUq 

+Fi  *(p,‘t)F2(p.‘l)e~j2°q  (4) 

where  the  superscript  asterisk  represents  the  complex  conju¬ 
gate  and  ( p,  q)  is  the  spatial  frequency  coordinate  system.  For 
simoiicity,  the  effects  of  the  magnification  and  the  DC  stop 
are  neglected  in  the  above  equauon.  It  is  evident  that  Equa- 
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tion  (4)  is  the  joint  power  spectral  disnbution  of  FA  p.  q>  and 
Fz(p.q).  Alternatively.  Equation  (4,  ^_i  be  written  as 

i{p<  q) -if, (/>.*)!* -!*(/>.<?)!' 

-2\Fl{p,q)\\F2(p,q)kos[2aq  f  A6(  p.q)] 

(-) 


where 


±<t>(p,q)  ~  4n(p,q)  -  <t>2(p,q)  (6) 

and  <t>\ip,q)  and  4^(p,q)  are  the  phase  distribution  of 
Fx(p,q)  and  F:(p,q),  respectively.  It  is  however  interesting 
to  note  that  if  the  input  objects  are  idenucal.  i.e..  fx(x,  y)  = 
/2(x,  >•),  then  Equation  (5)  can  be  reduced  to  the  form 

I(p,q)  =  2| /;(/., «)|‘(1  4-  cos[?  iq\)  (7) 

where  the  cosme  term  represents  uniform  interference  fringes 
of  the  joint  power  spectral  distnbution.  On  the  other  hand,  if 
the  input  objects  are  different.  A<f>(  p,  q)  is  not  zero  and  it  is  a 
rather  complicated  function.  The  cosme  factor  in  Equation  (5) 
no  longer  provides  a  stnct  uniform  fringe  pattern.  Thus  it  will 
eventually  produce  lower  correlation  peaks  through  the  in¬ 
verse  Founer  transformation. 

The  fringe  pattern  captured  by  the  CCD  camera  can  be  fed 
into  a  microcomputer  for  binanzation  with  a  thresholding 
procedure.  The  binarized  fringe  pattern,  in  principle,  can  be 
written  onto  a  MOSLM  to  obtain  a  binarized  phase  only 
fringe  pattern  (5],  such  that  the  transmittance  function  of  the 
MOSLM  would  be 


n(p>q) 


j  e+i* 

\e~J” 


I(p<q)  a  b 
l(p,q)  <  It 


(8) 


here  1T  is  the  binarizing  threshold  level.  In  reality,  the 
interference  fringe  structure  has  a  Gaussian  shaped  pedestal 
component.  A  stop  with  a  ring  window  would  pass  only  the 
uniform  pedestal  fringe  pattern.  Then  the  lowest  end  of  the 
dynamic  range  of  the  A/D  converter  for  the  video  signal  may 
be  selected  as  the  threshold  level.  Since  e-J*  =  cost  ±»)  *» 
±  1,  Equation  (8)  can  also  be  wntten  as 


H{p,q) 


l(p,q)  it  lT 

l(p<q)  <  It 


(9) 


If  the  input  objects  are  identical,  then  H(p,q)  reduces  to  a 
one-dimensional  bipolar  rectangular  function  with  a  spatial 
frequency  equal  to 


la 

f  =T\  (10) 

where  f  is  the  focal  length  of  the  transform  lens  and  X  is  the 
illuminating  wavelength.  Thus  H(  p,  q)  can  be  expanded  into 
a  Founer  cosme  series  as  given  by 

H(p,q)“  £  bnCO$[2anq]  (11) 

n— l 

where  bn  are  the  Fourier  coefficients  of  H(p,q). 


If  the  binary  phase  distnbution  of  Equauon  (11)  is  read 
out  by  coherent  illummauon.  as  shown  in  Figure  1.  the  output 
complex  light  distribution  can  be  shown  to  be 

£(*->■)“:  E  bno{y-lan)  b_n~b„  (12) 

rt—  —  *» 

This  yields  a  senes  of  spots.  The  presence  of  a  first  order  spot 
indicates  a  match.  On  the  contrary,  if  /,(*,  y)  differs  from 
f2(x,  y),  the  spot  would  be  absent.  Thus  we  see  that  a  binary 
phase  only  joint  transform  correlator  can  be  used  for  optical 
pattern  recognition. 

In  the  following,  we  should  provide  computer  simulation 
tests  for  the  proposed  binary  joint  transform  correlator 
(BJTO.  The  128  X  128  pixel  reference  and  target  images  are 
loaded  from  disk  files  and  written  onto  the  upper  and  the 
lower  halves  of  the  input  plane.  We  assume  that  the  A/D 
converter  for  the  video  signal  has  eight  bit  resolution.  To 
avoid  the  conversion  error,  we  may  discard  the  least  signifi¬ 
cant  bit.  The  threshold  level  was  chosen  to  be  the  lowest  end 
of  the  dynamic  range  of  the  A/D  converter  such  as 

(U) 

Figure  2(a)  shows  a  reference  image.  Figures  2(b)  and  (c) 
represent  target  images.  Figure  2(b)  was  obtained  by  adding 
an  additive  Gaussian  noise  to  the  reference,  with  a  signal-to- 


(a) 


(b) 


(c) 

Figure  2  Input  images:  (a)  reference  image  of  a  tank:  (b)  target 
embedded  in  noise  (SNR  =  i  dB):  (c)  target  image  of  a  truck 
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(a)  (b)  (c) 


Figure  3  Binarized  interference  fringes.  The  DC  component  is  eliminated  by  a  circular  stop,  (a)  Noiseless  case  of  Figure  2(a);  (b)  noisy 
case  of  Figure  2(b);  (c)  different  objects  of  Figures  2(a)  and  (c) 


;! 


Figure  4  Output  correlation  peaks:  (a)  noiseless  case  of  Figure  3(a).  (b)  noisy  case  of  Figure  3(b);  (c)  different  objects  of  Figures  2(a) 
and  (c) 


noise  ratio  (SNR)  equal  to  1  dB,  Figure  3  shows  the  computer 
simulated  binarized  joint  transform  power  spectrum  for  noise¬ 
less,  noisy,  and  different  objects.  The  SNR  for  the  noisy  target 
is  about  1  dB.  A  uniform  spatial  frequency  fringe  pattern  can 
readily  be  seen  in  Figure  3(a).  However,  uniformity  of  the 
fringes  is  somewhat  deteriorated  under  the  noisy  environment 
as  can  be  seen  in  Figure  3(b).  No  clear  fringes  can  be  observed 
in  Figure  3(c).  The  corresponding  output  correlations,  simu¬ 
lated  by  digital  computer,  are  shown  in  Figure  4.  A  compari¬ 


son  between  Figures  4(a)  and  (b)  shows  that  the  correlation 
peaks  reduce  to  about  a  half  for  an  input  SNR  =  1  dB.  As  is 
shown  in  Figure  4,  no  correlation  peak  was  obtained  for  the 
cross  correlation  of  different  images. 

In  conclusion,  we  have  investigated  a  binary  phase  only- 
joint  transform  optical  correlator  for  optical  pattern  recogni¬ 
tion.  The  major  advantage  of  the  proposed  system  must  be  the 
avoidance  of  synthesizing  a  spatial  matched  filter.  The  combi¬ 
nation  of  CCD  camera  and  MOSLM  may  have  an  advantage 
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over  the  optically  addressed  SLM  such  as  flexibility  of  thresh¬ 
old  level  selection.  We  have  shown  that  high  correlation  peaks 
can  be  readily  obtained  under  the  noisy  environment. 

We  acknowledge  the  support  of  U.  S.  Army  Research  office 
contract  No.  DAAL03-87-0147. 
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